Method and system for rule based display of sets of images using image content derived parameters

ABSTRACT

The invention provides, in some aspects, a system for implementing a rule derived basis to display image sets. In various embodiments of the invention, the selection of the images to be displayed, the layout of the images, as well as the rendering parameters and styles can be determined using a rule derived basis. The rules are based on meta data of the examination as well as image content that is being analyzed by neuronal networks. In an embodiment of the present invention, the user is presented with images displayed based on their preferences without having to first manually adjust parameters.

PRIORITY CLAIM

This application is a continuation of (1) U.S. application Ser. No.16/531,436 filed Aug. 5, 2019, which is a continuation of (2) U.S.application Ser. No. 16/052,571 filed Aug. 1, 2018, which is acontinuation in part of (3) U.S. application Ser. No. 15/870,681 filedJan. 12, 2018 which issued as U.S. Pat. No. 10,373,368 on Aug. 6, 2019,which is a continuation of (4) U.S. application Ser. No. 15/380,848filed Dec. 15, 2016 which issued as U.S. Pat. No. 9,898,855 on Feb. 20,2018, which is a continuation of (5) U.S. application Ser. No.14/611,163 filed Jan. 30, 2015 which issued as U.S. Pat. No. 9,524,577on Dec. 20, 2016, which is a continuation of (6) U.S. application Ser.No. 13/831,975 filed Mar. 15, 2013 which issued as U.S. Pat. No.8,976,190 on Mar. 10, 2015. This application also claims the benefit ofpriority of (7) U.S. Provisional Patent Application Ser. No. 62/562,460,filed Sep. 24, 2017, (8) U.S. Provisional Patent Application Ser. No.62/576,587, filed Oct. 24, 2017, and (9) U.S. Provisional PatentApplication Ser. No. 62/712,912, filed Jul. 31, 2018. The teachings of(1)-(9) are herein incorporated by reference in their entireties and forall purposes.

BACKGROUND OF THE INVENTION

In order to diagnose a traditional X-Ray examination, the images printedon films would be ‘hung’ in front of a light box. For multi-imageexaminations, as well as for comparison with priors, the ‘hanging’ wouldoften follow a specific protocol. For example, a particular organizationor doctor may choose for a two-view chest X-Ray with a two-view priorexam, that the films be hung from left to right as follows: Frontal viewof current examination, lateral view of current examination, frontalview of prior examination, lateral view of prior examination. Incontrast, the doctor may hang mammography exams with the correspondingviews of current and prior next to each other, if that was moreappropriate for the diagnostic workflow in that case. Thus, theorganization or doctor developed a traditional ‘Hanging Protocol’.Currently, the film and the light box are often being replaced bycomputer systems, called PACS (Picture Archiving and CommunicationSystem). PACS systems can mimic the Hanging Protocols.

Traditional X-Ray examinations typically produce one or a small numberof single two dimensional (2D) images. In contrast, the more advancedimaging modalities such as Computer Tomography (CT), Magnetic ResonanceImaging (MRI) or Positron Emission Tomography (PET) can produce dozensof series, each consisting of a hundred or more images. It is possibleand not uncommon to review images from these advanced modalities in thesame manner as traditional X-Ray images, i.e., by hanging the individualimages side-by-side, either on a light-box or using a PACS system.

SUMMARY OF THE INVENTION

The invention pertains to digital data processing and, moreparticularly, by way of example, to the visualization of image data.Three dimensional (3D) and four dimensional (4D) image data is routinelyacquired with CT, MRI, PET, confocal microscopes, 3D ultrasound devices,and other imaging devices. The medical imaging market is just oneexample of a market that uses these devices. The visualization of imagedata market is growing rapidly, with new CT scanners collecting largeramounts of data more quickly than previous generation CT scanners. Theinvention has application to areas including medical imaging,atmospheric studies, astrophysics and geophysics.

With the rapid increase in the amounts and types of information that canbe acquired using imaging technology, we have identified a substantialproblem with integrating different types of image-based information intoa form that can be used by a diagnostician, for example a physician.Namely, although there may be many different types of image data, theforms, formats, integration, and display of relevant information isextremely difficult for a person to carry out without sophisticatedcomputer processing. Embodiments of this invention therefore provide acomputer-based analytic framework whereby image-based information from avariety of different sources can be integrated to provide increasedability to display relevant information, e.g., to display informationfor a physician to diagnose and evaluate a patient's condition. We haveidentified another substantial problem in the art, namely the increasedlikelihood of confusion of image-based information from differentproblems, e.g., a physician can incur increased likelihood of confusionof image-based information from different patients. In such situations,a diagnostician (e.g., physician) may be presented with image-basedinformation from different patients. Such inadvertent conflation canproduce misdiagnosis or mistaken non-diagnosis. In each case, theoutcome can be serious, e.g., misdiagnoses of a patient can result inincreased chance of morbidity and/or mortality.

In general aspects of this invention, a First Study is first selectedfor review by a physician or diagnostician. Selection of a Study willgenerally be based on some particular characteristic. Suchcharacteristic can be anatomical, disease-based, or both. Once a FirstStudy is selected, an Additional Candidate Study can be selected basedon the anatomical location of the First Study. Therefore, if the FirstStudy is a Chest X-Ray, an Additional Candidate Study can be a Chest CTscan, MRI, positron-emission tomography (PET) scan, or other image ofthe chest. Alternatively, if a First Study is an X-Ray image of thegastrointestinal tract, an Additional Candidate Study could be a seriesof X-Ray images taken after ingestion of a contrast agent (such asbarium). It can be appreciated that such anatomically selectedAdditional Candidate Studies can be applied to any organ, organ system,or tissue.

Alternatively, Additional Candidate Studies can be selected based on thetype of disorder or disease being evaluated. For example, in a case inwhich a patient has had a diagnosis of cancer of one organ (e.g., lung),it can be desirable for Additional Candidate Studies to be targeted toidentification of metastases in another organ. Thus, if a First Study isa Chest X-Ray, an Additional Candidate Study can be of the lymphaticsystem, head and neck, or various abdominal quadrants. Such AdditionalCandidate Studies may be X-ray, CT scans, MRI scans, PET scans, vascularvisualizations (e.g., with injected contrast media) or histologicalimages taken during a biopsy. Because the degree of detail (i.e.,“granularity”) obtained using different imaging techniques may varywidely it can be desirable to have a Rule Based process whereby thegranularity of an Additional Candidate Study is increased over that ofthe First Study.

For example, a Chest X-Ray is a two-dimensional image in which theentirety of the chest and lungs is represented as a flat image. AnAdditional Candidate Study could be a CT scan, where “2-dimensional”images are acquired at a series of different “depths” (e.g., “slices”)through the organ. If the 2-dimnensional images are of sufficientquality to produce a 3-dimensional image of the organ with desirabledegree of granularity, then the Additional Candidate Study can bedepicted and displayed along with the image of the First Study.

General Rule 1 for selecting an Additional Candidate Study therefore canbe:

-   -   IF (Primary.Dicom.BodyPartExamined is “ANATOMICAL REGION 1”, and        Primary.Dicom.Modality=IMAGE TYPE 1”)    -   THEN SELECT other studies for loading, WHERE        (Other.Dicom.BodyPart Examined=ANATOMICAL REGION 1” and        Other.Dicom.Modality=“IMAGE TYPE 2”).

If desired, in General Rule 1, Additional Candidate Studies can target“Other.Dicom.Modality=“IMAGE TYPE 2”).

It can be appreciated that any number of Additional Candidate Studiescan be integrated using the computer-based processes of this invention.

Alternatively, General Rule 2 for selecting an Additional CandidateStudy therefore can be:

-   -   (Primary.Dicom.Disease is “DISEASE 1”, and        Primary.Dicom.Modality=IMAGE TYPE 1”)    -   THEN SELECT other studies for loading, WHERE        (Other.Dicom.Disease=“DISEASE 1” and        Primary.Dicom.Modality=“IMAGE TYPE 2”).

It can be readily appreciated that application of General Rule 2 canintegrate other Anatomical Regions and a number of different ImageTypes.

In an embodiment of the present invention, a method or system uses arule derived basis to display image sets. In various embodiments of thepresent invention, the selection of the images to be displayed, thelayout of the images, i.e., the hanging, as well as the renderingparameters and styles can be determined using a rule derived basis. Inan embodiment of the present invention, the user is presented withimages displayed based on their preferences without having to firstmanually adjust parameters. Accordingly, there is a time saving in notdisplaying images initially in a non-rule derived basis.

The parameters used in the rules can be derived from meta data stored inthe data files, such as the DICOM parameters, but they can also bederived from the image content using one or more Convolutional NeuralNetworks (CNN). Each CNN is pre-trained to derive relevant aspects aboutthe image. At the time of data ingestion, the CNN is applied to theimages of the Study, and the output of the CNN is used to define ImageContent based parameters. Examples for such Image Content BasedParameters are (i) finer granular anatomic information, e.g. whether ornot a particular organ is covered by a particular study, or (ii) whetheror not a particular medical condition is present, such as a fracture orbleeding.

These and other aspects of the invention are evident in the drawings andin the description that follows.

BRIEF DESCRIPTION OF THE DRAWINGS

This invention is described with respect to specific embodimentsthereof. Additional features can be appreciated from the Figures inwhich:

FIG. 1 depicts a flow chart showing the steps of applying various rulesto the selected Study, according to an embodiment of the invention;

FIG. 2 depicts the resulting display for an example study, according toan embodiment of the invention; and

FIG. 3 shows an example of a user interface to specify rules including adialog box to configure Study Selection rules, according to anembodiment of the invention.

FIG. 4 depicts a line drawing of an illustration of the human spine,with the vertebrae labeled according to standard terminology in humananatomy.

FIG. 5A is a line drawing of an illustration of Study S1 of a differentpart of the spine that may have been taken at different point in time toFIGS. 5B-5D. Study S1 is a scan of the lumbar spine containing alllumbar vertebrae L1, L2, L3, L4, and L5.

FIG. 5B is a line drawing of an illustration of Study S2 of a differentpart of the spine that may have been taken at different point in time toFIGS. 5A, 5C, and 5D. Study S2 is a scan of the cervical spine and doesnot contain any lumbar vertebrae.

FIG. 5C is a line drawing of an illustration of Study S3 of a differentpart of the spine that may have been taken at different point in time toFIGS. 5A, 5B, and 5D. Study S3 is a scan of vertebrae extending fromlumbar to thoracic spine and also contains all five lumbar vertebrae(L1, L2, L3, L4, and L5).

FIG. 5D is a line drawing of an illustration of Study S4 of a differentpart of the spine that may have been taken at different point in time toFIGS. 5A-5C. Study S4 is a scan of the thoracic spine but it alsocontains lumbar vertebrae L1 and L2.

FIG. 6A is a line drawing corresponding to FIG. 5A and depicts theresult of applying a particular neuronal network to Study S1 taken fromone patient at a specific time, depicting different sections of thespine. The Image Content Based Parameter computed by the neuronalnetwork in this example is the set (list) of vertebrae shown below thearrow, according to an embodiment of the present invention.

FIG. 6B is a line drawing corresponding to FIG. 5B and depicts theresult of applying a particular neuronal network to Study S2 taken fromone patient at a specific time, depicting different sections of thespine. The Image Content Based Parameter computed by the neuronalnetwork in this example is the set (list) of vertebrae shown below thearrow, according to an embodiment of the present invention.

FIG. 6C is a line drawing corresponding to FIG. 5C and depicts theresult of applying a particular neuronal network to Study S3 taken fromone patient at a specific time, depicting different sections of thespine. The Image Content Based Parameter computed by the neuronalnetwork in this example is the set (list) of vertebrae shown below thearrow, according to an embodiment of the present invention.

FIG. 6D is a line drawing corresponding to FIG. 5D and depicts theresult of applying a particular neuronal network to Study S4 taken fromone patient at a specific time, depicting different sections of thespine. The Image Content Based Parameter computed by the neuronalnetwork in this example is the set (list) of vertebrae shown below thearrow, according to an embodiment of the present invention.

FIG. 7 depicts a subset of the DICOM tags and Image Content BasedParameters extracted from the Studies S1, S2, S3, S4 shown in FIG. 8,namely Modality, BodyPartExamined, and Vertebrae, according to anembodiment of the present invention.

FIG. 8 depicts an example for a Study Selection Rule according to anembodiment of the present invention. The Rule uses the Image ContentBased Parameter Vertebrae. The table shows the result of the selectionif Study S1 was loaded by a user as primary study. Studies S3 and S4would be selected for comparison, because they have common anatomy withthe primary study, and Study S2 would not be selected. As is obvious inthis example, this could not be achieved using a rule based on the DICOMtag BodyPartExamined alone.

FIG. 9A is a line drawing of a current study which has been loaded by auser into an exemplary hanging protocol showing FIGS. 9A-9D.

FIG. 9B is a line drawing of a current study which has been loaded by auser into an exemplary hanging protocol showing FIGS. 9A-9D.

FIG. 9C is a line drawing of a prior study which has been loaded into anexemplary hanging protocol showing FIGS. 9A-9D, where FIG. 9C is one oftwo series of relevant prior studies identified by a Study SelectionRule as containing different but overlapping parts of the anatomy of thesame patient, according to an embodiment of the present invention.

FIG. 9D is a line drawing of a prior study which has been loaded into anexemplary hanging protocol showing FIGS. 9A-9D, where FIG. 9D is one oftwo series of relevant prior studies identified by a Study SelectionRule as containing different but overlapping parts of the anatomy of thesame patient, according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Definitions

The transitional term “comprising” is synonymous with “including,”“containing,” or “characterized by,” is inclusive or open-ended and doesnot exclude additional, unrecited elements or method steps.

The transitional phrase “consisting of” excludes any element, step, oringredient not specified in the claim, but does not exclude additionalcomponents or steps that are unrelated to the invention such asimpurities ordinarily associated with a composition.

The transitional phrase “consisting essentially of” limits the scope ofa claim to the specified materials or steps and those that do notmaterially affect the basic and novel characteristic(s) of the claimedinvention.

The term “Study” will be used to refer to the set of images produced byan examination. A Study consists of one or more images. The images canbe grouped into one or more image series. Each image, each series, andthe whole Study can have different parameters attached. For medicalimages these can be defined by the Digital Imaging and Communication inMedicine (DICOM) standard.

Some or all of the images in a Study can form one or more threedimensional “Volumes.” For 3D modalities, such as CT or MRI, often eachindividual image in the series corresponds to a volume, but that is nota requirement. For example a cardiac CT may contain multiple 3D volumescovering the heart, each corresponding to a different point in thecardiac cycle, and all of the images belonging to all of these volumesbeing grouped into the same series.

The term “Hanging Protocol” will be used to refer to specificconventions how X-Ray films are arranged (hung) at a light box.

The term “Display Protocol” will be used to refer to the way images aredisplayed in a computer system, specifically the selection of the imagesto be displayed, the layout of the images, as well as the renderingparameters and styles.

The term “View” will be used to refer to data corresponding to a digitalimage view of a Set of Images rendered with a given set of renderingparameters and rendering modes.

The term “Viewport” will be used to refer to the logical part of thescreen on the client computer in which a particular View is displayed,for example the user interface on the client computer can contain fourrectangular Viewports 1160 of which three show a frontal, left, andbottom view respectively of a particular data, while the fourth viewermight show a 2D cross section through the same or a different data set.

The term “Sets of Images” or “Image Set” will be used to refer to one ormore images, selected based on the rules.

The term “Study Selection Rules” will be used to refer to the rules usedto select the studies to be displayed.

The term “Protocol Selection Rules” will be used to refer to the rulesused to select the layout of the images to be displayed.

The term “Image Set Rules” will be used to refer to the rules used toform Image Sets 1165 from the images of one or more Study by applyingselection, sorting, and breaking rules.

The term “Style Rules” will be used to refer to the rules to determinewhich rendering type, rendering style, and rendering parameters are usedfor a particular Image Set 1165 in a particular viewer.

The term “Volume Rendering” will be used to refer to Volume Renderingtechniques including shaded Volume Rendering techniques, maximumintensity projection (MIP), oblique slicing or multi-planar reformats(MPR), axial/sagittal and coronal slice display, and thick slices (alsocalled slabs). In medical imaging, for example, Volume Rendering is usedto display 3D images from 3D image data sets, where a typical 3D imagedata set is a large number of 2D slice images acquired by a CT or MRIscanner and stored in a data structure.

The term “anatomical characteristic” will be selected from the groupconsisting of one or more of spine, chest, abdomen, breast, shoulder,trapezius, arm, elbow, wrist, finger, pelvis, hip, fibula, knee, tibula,ankle, foot, neck, head, temporomandibular junction, face, brain,dentition, sinus, adrenals, retina, pituitary, and prostate. Theanatomical characteristic can include the Body Part Examined. Ananatomical characteristic can be either natural or pathologic. A naturalanatomical characteristic of a patient would be the presence of sevencervical vertebrae. A pathologic anatomical characteristic of a patientwould be the presence of only six cervical vertebrae.

The term “anatomical feature” refers to a medical condition, e.g.,whether a fracture or bleeding is present in a given image or volume. Ananatomical feature can be a fractured fibula, a herniated disc, urethralbleeding, e.g. bleeding with benign prostate hyperplasia, laceratedbreast, Gun Shot Wound (GSW) to the chest, infection by Treponemapertenue giving rise to YAWS lesion in left distal leg. An anatomicalfeature is pathologic. In an embodiment of the invention, if ParameterAis fibula then the anatomical feature can be a fractured fibula.

The term “disease based characteristic” can be selected from the type ofdisorder or disease being evaluated, e.g., a diagnosis of lung cancer.The disease based characteristic can include the Body Part Examined. Adisease based characteristic is pathologic.

The phrase “carried out using Convolutional Neural Networks” means thatCNN is used to select or identify based on an anatomical characteristicParameterZ. For example, when ParameterZ is SPINE, other secondarystudies with the same anatomical characteristic can be selected and CNNcan generate one or more ImageContentBased parameters from on eor moreof these secondary studies. The presence of the ImageContentBasedparameters can be used to generate a final list for display. This waythe rule would not select a cervical spine scan for comparison when thecurrent study is a lumbar spine, but it could select a prior thoracicspine scan for comparison, if that scan did have an overlap with thecurrent scan of the lumbar spine.

Overview

Often, the traditional ‘Hanging Protocol’ is either not intuitive,cannot display the information in a manner in which it can be reviewedor is not the most efficient way to display images. Alternative ways ofrendering the acquired images can be more efficient or more appropriatefor displaying the information. Examples include Volume Renderingtechniques or maximum intensity projections of stacks of cross-sectionalimages, rendering of oblique slices, rendering of thick slices or slabs,or rendering of fused images (e.g. in PET/CT). Specialized diagnosticworkstations that are often specific to a clinical application area areused to provide appropriate rendering of the acquired images. Asorganizations and doctors require better and faster visualizationmethods that allow users to interact with the image data in real-time,the requirements and demands for displaying the data will increase.

FIG. 2 depicts an example study where the rules have created two Sets ofImages. One Set of Images consists of a series of CT images forming a 3Dvolume, which is depicted in a volume rendered style in the Viewport1160 in the upper left and in three orthogonal cross sections in thethree other viewports in the left half of the screen. The second Set ofImages consist of one chest X-Ray, assigned to a single Viewport 1160covering the right half of the screen and rendering the X-Ray in 2Dstyle. Appropriate data windows have been chosen by the rules tohighlight the vasculature in the 3D rendering, as this is a study withcontrast, as the rules can determine by the StudyDescription containingthe word ‘contrast’.

FIG. 1 is a flow chart showing how the rules are used to create the twoSets of Images shown in FIG. 2. As shown in FIG. 1, a primary Study 1105which can be manually selected by a user. In step (i) 1120, based onStudy Selection Rules 1115 which interrogate parameters in the primaryStudy 1105 such as DICOM Parameters and Abstract Tags of both theprimary Study 1105 and the candidate studies 1110, the Study SelectionRules 1115 can identify additional candidate studies 1110. The secondset of studies 1125 which includes the candidate studies 1110 and theprimary Study 1105 are available to be loaded into Viewports 1160. Instep (ii) 1140, the Protocol Selection Rules 1135 select a DisplayProtocol 1145 from the Available Display Protocols 1130 based on DICOMParameters and Abstract Tags present in the second studies 1125. In step(iii) 1155, Image Set Rules 1150 are used to define a plurality of ImageSets 1165 from the second studies 1125. The one or more Viewports 1160are defined in the Display Protocol 1145. In step (iv) 1175, ViewportAssignment Rules 1170 assign one or more Image Sets 1165 to one or moreViewports 1160. In step (v) 1185, Style Rules 1180 define a renderingstyle and rendering parameters. In an embodiment of the invention steps(i) through (v) are performed by a server processor running a renderserver program with an interface shown in FIG. 3 in which the rules(Study Selection Rules 1115, Protocol Selection Rules 1135, Image SetRules 1150, Viewport Assignment Rules 1170, and the one or more StyleRules 1180) are used to automatically select and display the Image Sets1165 in the Viewports 1160.

A render server program is described in U.S. application Ser. No.13/831,967, entitled “Multi-User Multi-GPU Render Server Apparatus andMethods”, inventors M. Westerhoff et al., which was filed Mar. 15, 2013,is herein expressly incorporated by reference in its entirety. A rulebased render server program is described in U.S. application Ser. No.13/831,982, entitled “Method and System for Transferring Data to ImproveResponsiveness when Sending Large Data Sets”, inventors D Stalling etal., which was filed Mar. 15, 2013, is herein incorporated by referencein its entirety.

The system can be connected to a network, e.g. in a hospital, with databeing sent to the system from Imaging Modalities, such as CT Scanners oran X-Ray machine, from other computer systems, such as an image archiveor PACS system, e.g. using the DICOM network protocol and file format orother suitable network protocols, such as HTTP, HTTPS, SMB and othersuitable file formats, such as TIFF, PNG, JPEG. Data can also beinserted into the system by using a CD or DVD, or a USB Memory Stick orother portable media. The system can also query other systems, such asan image archive, and retrieve data, using suitable network protocolsand file formats, such as DICOM, or WADO.

We refer to the process of a new imaging study being sent to orretrieved by the system as “Study Insertion” in the following.

At the time of Study Insertion for each Study, the images and thevolumes of the Study are being processed individually by one or moreConvolutional Neural Network (CNN). Separate CNNs can be used for imagesand volumes respectively, and pre-selection rules can be used todetermine which images or volumes to process with which CNN. Forexample, the DICOM tag Modality can be used to process CT images with adifferent CNN than MRI images. The term “Study Selection Parameters”will be used to refer to one or more parameters chosen from the group ofDICOM Parameters, Abstract Tags, and Image Content Based Parameters.

The term “Convolutional Neural Network,” “CNN,” or the like refer, inthe usual and customary sense, to a class of deep, feed-forwardartificial neural networks that has successfully been applied toanalyzing e.g., visual imagery. Exemplary references disclosing methodsand systems for CNN include: Alex Krizhevsky et al., ImageNetClassification with Deep Convolutional Neural Networks, In: ADVANCES INNEURAL INFORMATION PROCESSING SYSTEMS 25 (Eds. F. Pereira, C. J. C.Burges, L. Bottou and K. Q. Weinberger), Curran Associates, Inc., 2012,pp. 1097-1105; and Christian Szegedy et al, Going Deeper withConvolutions, In: COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2015,each of which is incorporated herein by reference and for all purposes.

The parameters used in the rules can be derived from an image contentusing one or more CNN. In an embodiment of the present invention, CNNcan be used in the context of localization and object detection. In anembodiment of the present invention, a CNN consists of an input layer,one or more hidden layers and an output layer. In an embodiment of thepresent invention, optimizing the performance of a CNN can beaccomplished by increasing the depth or the number of levels of thenetwork and its width or the number of units at each level. In anembodiment of the present invention, the width defines the region ofspace within which visual stimuli affect the firing of a single neuronor the receptive field. Given the availability of a large amount oflabeled training data it is possible to train higher quality models.However, increased layers and/or widths typically means a larger numberof parameters, which makes the enlarged CNN prone to overfitting, andincreased use of computational resources. In an embodiment of thepresent invention, the depth and width of the CNN can be maximized,while constraining the computational requirement. In an embodiment ofthe present invention, an additional 1×1 convolutional layers can beadded to the receptive field. In an alternative embodiment of thepresent invention, an additional n×n convolutional layers can be addedto the receptive field. In an embodiment of the present invention,filters can be used to reduce the dimension and thereby constraincomputational demands. In an alternative embodiment of the presentinvention, the outputs of multiple nodes at one layer can be combinedinto a single node in the next layer to constrain the computationaldemands. In an embodiment of the present invention, a resulting matrixof the CNN would include sparse clustering between regions of denseclustering. In an embodiment of the present invention, reconfiguring amatrix containing sparse clustering between regions of dense clusteringinto two or more relatively dense submatrices can be used to constrainthe computational demands. In an embodiment of the present invention,max-pooling in which a matrix is partitioned into a set ofnon-overlapping submatrices and the maximum for each submatrix is outputcan be used to constrain the computational demands. In an embodiment ofthe present invention, filtering is followed by rectified linearactivation. In an embodiment of the present invention, if theprobability distribution of the data-set is representable by a large,very sparse CNN, then the optimal network topology can be constructedlayer by layer by analyzing the correlation statistics of theactivations of the previous layer and clustering neurons with highlycorrelated outputs and the institution of multi-scale processing. In anembodiment of the present invention, each CNN is pre-trained to produceone or more output channels that represent relevant aspects of the inputimages or volumes. These output channels of the CNNs are referred to as“Image Content Based Parameters” in the following. At the time of dataingestion, the CNN is applied to the images of the Study. In anembodiment of the present invention, the output of the CNN is used todefine Image Content based parameters. In an embodiment of the presentinvention, an Image Content Based Parameter includes finer granularanatomic information. In an embodiment of the present invention, anImage Content Based Parameter includes whether a particular organ iscovered by a particular study. In an embodiment of the presentinvention, an Image Content Based Parameter includes whether aparticular medical condition is present. In an embodiment of the presentinvention, an Image Content Based Parameter includes a fracture. In anembodiment of the present invention, an Image Content Based Parameterincludes a fracture of a specific bone. In an embodiment of the presentinvention, an Image Content Based Parameter includes a fracture of atibia. In an embodiment of the present invention, an Image Content BasedParameter includes bleeding. In an embodiment of the present invention,an Image Content Based Parameter includes arterial bleeding. In anembodiment of the present invention, an Image Content Based Parameterincludes arterial bleeding. In an embodiment of the present invention,an Image Content Based Parameter includes external venal bleeding. In anembodiment of the present invention, an Image Content Based Parameterincludes internal venal bleeding. In an embodiment of the presentinvention, an Image Content Based Parameter includes venal bleeding. Inan embodiment of the present invention, an Image Content Based Parameterincludes external venal bleeding. In an embodiment of the presentinvention, an Image Content Based Parameter includes internal venalbleeding. In an embodiment of the present invention, an Image ContentBased Parameter includes varicose internal venal bleeding. As can beappreciated by a person of ordinary skill Image Content Based Parametercan cover a variety of medical conditions and their anatomic locations.

FIG. 4 depicts an illustration of the human spine, with the vertebrae(C1 130, C2 135, C3 140, C4 145, C5 150, C6 152, C7 154, Th1 156, Th2158, Th3 160, Th4 162, Th5 164, Th6 166, Th7 168, Th8 170, Th9 172, Th10174, Th11 176, Th12 178, L1 180, L2 182, L3 184, L4 186, L5 188, Ossacrum 190 and coccyx 191) labeled according to standard terminology inhuman anatomy. Note that any labels and any grayscale coding (405corresponds with cervical vertebrae, 410 corresponds with thoracicvertebrae, 415 correspond with lumbar vertebrae, 420 corresponds withsacrum and 425 corresponds with the coccyx) in FIG. 4, and FIGS. 5A-5Dare schematic representations and are not present in any originalimages. FIG. 5A depicts Study S1 , FIG. 5B depicts Study S2, FIG. 5Cdepicts Study S3, and FIG. 5D depicts Study S4 showing different partsof the spine that may have been taken at different time points. FIG. 5Adepicts an illustration of Study S1, a scan of the thoracic—sacrumregion of the spine containing a portion of a thoracic vertebra 178, alllumbar vertebrae L1 180, L2 182, L3 184, L4 186, and L5 188, and aportion of the sacrum 190. FIG. 5B depicts an illustration of Study S2,a scan of the cervical spine showing cervical vertebrae C1 (Atlas) 130,C2 (Axis) 135, C3 140, C4 145, C5 150, C6 152, C7 154, Th1 156, and aportion of Th2 158. FIG. 5B does not contain any lumbar vertebrae. FIG.5C depicts an illustration of Study S3, a scan of vertebrae extendingfrom lumbar to sacral regions of the spine including a portion of C7154, Th1 156, Th2 158, Th3 160, Th4 162, Th5 164, Th6 166, Th7 168, Th8170, Th9 172, Th10 174, Th11 176, Th12 178, L1 180, L2 182, L3 184, L4186, L5 188, and portion of Os sacrum 190. FIG. 5C contains all fivelumbar vertebrae (L1 180, L2 182, L3 184, L4 186, and L5 188). FIG. 5Ddepicts an illustration of Study S4, a scan of the thoracic and lumbarregions of the spine including a portion of C7 154, Th1 156, Th2 158,Th3 160, Th4 162, Th5 164, Th6 166, Th7 168, Th8 170, Th9 172, Th10 174,Th11 176, Th12 178, L1 180, L2 182 and a portion of L3 184. In anembodiment of the invention, applying a Convolutional Neuronal Network(CNN) to Study S1 results in FIG. 6A which corresponds to FIG. 5A whichrecognizes L1 180, L2 182, L3 184, L4 186, and L5 188 in Study S1 andoutputs the Image Content Based Parameters {L1, L2, L3, L4, L5}. In anembodiment of the invention, applying a Convolutional Neuronal Network(CNN) to Study S2 results in FIG. 6B which corresponds to FIG. 5B whichrecognizes C1 (Atlas) 130, C2 (Axis) 135, C3 140, C4 145, C5 150, C6152, C7 154, Th1 156 in Study S2 and outputs the Image Content BasedParameters {C1, C2, C3, C4 C5, C6, C7, T1}. In an embodiment of theinvention, applying a Convolutional Neuronal Network (CNN) to Study S3results in FIG. 6C which corresponds to FIG. 5C which recognizes Th1156, Th2 158, Th3 160, Th4 162, Th5 164, Th6 166, Th7 168, Th8 170, Th9172, Th10 174, Th11 176, Th12 178, L1 180, L2 182, L3 184, L4 186, L5188 in Study S3 and outputs the Image Content Based Parameters {T1, T2,T3, T4 T5, T6, T7, T8, T9, T10, T11, T12, L1, L2, L3, L4 L5}. In anembodiment of the invention, applying a Convolutional Neuronal Network(CNN) to Study S4 results in FIG. 6D which corresponds to FIG. 5D whichrecognizes Th1 156, Th2 158, Th3 160, Th4 162, Th5 164, Th6 166, Th7168, Th8 170, Th9 172, Th10 174, Th11 176, Th12 178, L1 180, L2 182 inStudy S4 and outputs the Image Content Based Parameters {T1, T2, T3, T4T5, T6, T7, T8, T9, T10, T11, T12, L1, L2}. That is, based on the imagethe CNN recognizes vertebrae and outputs the Image Content BasedParameters. In an embodiment of the invention, these Image Content BasedParameters can then be used to select which of Studies S2, S3 and S4 canhelp a medical practitioner who has measured Study S1 make appropriatecomparisons and diagnoses. For example, since the CNN analysis of StudyS1 was able to recognize L1 180, L2 182, L3 184, L4 186, and L5 188 inStudy S1 and output Image Content Based Parameters {L1, L2, L3, L4, L5}the medical professional can be interested in displaying other studiesthat display the lumber vertebra L1, L2, L3, L4, L5.

As summarized in FIG. 7, the CNN analysis of Study S2 did not recognizelumber vertebra L1, L2, L3, L4, L5. In contrast, the CNN analysis ofStudy S3 did recognize lumber vertebra L1, L2, L3, L4, L5 and the CNNanalysis of Study S4 did recognize lumber vertebra L1, L2. As such theCNN analysis identifies Primary.ImageContentBasedParameter=“ParameterC”and Other.ImageContentBasedParameter=“ParameterD”, to be used in theStudy Selection Rule:

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        INTERSECTION(ParameterC,ParameterD) NOT EMPTY), which rule is        fulfilled for primary study S1, by study S3 and study S4, but        not study S2, provided that the primary study anatomical        characteristic is the same in study S3 and study S4.

Note that any labels and any grayscale coding (405 corresponds withcervical vertebrae, 410 corresponds with thoracic vertebrae, 415correspond with lumbar vertebrae, 420 corresponds with sacrum and 425corresponds with the coccyx) in FIG. 6 are not present in any originalimages. FIG. 6A shows that of the labeled vertebra 178, 180, 182, 184,186, 188, and 190 all lumbar vertebrae 180, 182, 184, 186, and 188appear as labeled by arrow. FIG. 6B shows that all of the labeledvertebra 130, 135, 140, 145, 150, 152, 154, 156, and 158 appear aslabeled by arrow. FIG. 6C shows that of the labeled vertebra 154, 156,158, 160, 162, 164, 166, 168, 170, 172, 174, 176, 178, 180, 182, 184,186, 188, and 190 appearing as labeled by arrow are 156, 158, 160, 162,164, 166, 168, 170, 172, 174, 176, 178, 180, 182, 184, 186, and 188.FIG. 6D shows that of the labeled vertebra 154, 156, 158, 160, 162, 164,166, 168, 170, 172, 174, 176, 178, 180, 182, and 184 appearing aslabeled by arrow are 156, 158, 160, 162, 164, 166, 168, 170, 172, 174,176, 178, 180, and 182. The Image Content Based Parameter computed bythe neuronal network in this example is the set (list) of vertebraeshown below the arrow. FIG. 8 depicts an example for a Study SelectionRule according to an embodiment of the present invention. The StudySelection Rule is as follows:

-   -   IF (Primary.Dicom.BodyPartExamined=“SPINE” and        Primary.Dicom.Modality=“MR”)    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.BodyPartExamined=“SPINE” and        (Other.Dicom.Modality=“MR”) AND        INTERSECTION(Primary.Vertebrae,Other.Vertebrae) NOT EMPTY).        The Rule uses the Image Content Based Parameter ‘Vertebrae’.        FIG. 8 shows the result of the selection if Study S1 was loaded        by a user as primary study. Studies S3 and S4 would be selected        for comparison, because they have common anatomy with the        primary study, and Study S2 would not be selected. As is obvious        in this example, this could not be achieved using a rule based        on the DICOM tag BodyPartExamined alone (e.g., LSPINE, SPINE and        TSPINE). FIG. 7 depicts a subset of the DICOM tags and Image        Content Based Parameters extracted from the Studies S1, S2, S3,        S4 shown in FIG. 8, namely Modality, BodyPartExamined, and        Vertebrae, according to an embodiment of the present invention.        FIGS. 9A-D depict an exemplary hanging protocol where the client        view is shown at 910, a menu including the functions ‘File’,        ‘View’, ‘Protocol’, ‘Tools’ and ‘Help’ is shown at 915, as        series of self-explanatory icons related to specific functions        is shown at 920, the number of series including ‘1 Series’, ‘2        Series’, ‘4 Series’, ‘6 Series’, ‘8 Series’, ‘12 Series’,        ‘Compare 1+1’, ‘Compare 2+2’, ‘Compare 4+4’, ‘Compare 6+6’,        ‘Multiplanar (act. series)’, ‘Multiplanar+3D MIP (act. series)’        to be viewed is selected at 925, where the identification of the        image displayed is shown at 930, the date of analysis of the        image displayed is shown at 940, the series identification and        image identification of the image displayed is shown at 950, and        the anatomical location descriptor (L=lateral) of the image        displayed is shown at 960, and the scale and magnification of        the view of the image displayed is shown at 970. FIGS. 9A and 9B        show two series of a current study that have been loaded by the        user and where a Study Selection Rule has identified a relevant        prior study containing a different but overlapping part of the        anatomy of the same patient and displays FIGS. 9C and 9D, two        series of the prior study for comparison (592 includes        diaphragm, 593 includes disks, 594 includes the posterior spinal        cord, 596 includes anterior spinal cord and vertebrae, 597        includes diaphragm and vertebrae and 598 includes the epithelial        layer), according to an embodiment of the present invention.

In embodiments, each CNN is pre-trained to produce one or more outputchannels that represent relevant aspects of the input images or volumes.These output channels of the CNNs are referred to herein as “ImageContent Based Parameters”. Image Content Based Parameters can beanatomical parameters. For example, they can be more fine granular thanthe information stored in DICOM parameters. For example the DICOMparameter BodyPartExamined may specify “SPINE”, whilst an Image ContentBased Parameters can be defined for each vertebrae, such as L1, L2, L3and so forth for the first, second, and third vertebrae in the lumbarspine. Image Content Based Parameters can also be defined for medicalconditions, such as whether a fracture or bleeding is present in a givenimage or volume.

The Image Content Based Parameters computed for a Study are stored in anappropriate form, e.g. in a database, a text file, or as private DICOMtags.

As is easily appreciated, Image Content Based Parameters can also becomputed at a later time than Study Insertion, e.g. by a scheduled taskonce every hour, or at the time of loading the study by the user.

Study Selection Rules 1115

In an embodiment of the present invention, based on the Study that theuser selects for display (primary Study 1105), the system can firstapply user defined rules to determine additional studies to be displayedtogether with the primary Study 1105. Such additional studies can beprior examinations that are relevant for the diagnosis of the currentStudy, or additional current studies. For example, a PET examinationwill often be looked at together with a CT examination acquired at thesame time. The set of rules are constructed as follows:

Each rule consists of a matching criterion for the primary Study 1105(primary condition), as well as matching criteria for additional studies(secondary condition). The matching criterion is an expressionconsisting of operators that allow evaluating the parameters of theStudy and comparing them to defined values. The parameters of the Studycan be any parameters defined by the DICOM standard, such as StudyDescription, Study Date, Modality, Patient Age, as well as any otherparameters that can be derived from the DICOM parameters or from theStudy itself, such as number of images, or number of image series aswell as Image Content Based Parameters. The operators are numeric orstring based operators, such as equals, greater than, less than,contains, etc. Expressions can be combined using Boolean operators suchas AND, OR, NOT. Operators can also contain more complex expressions,including user defined functions defined in an appropriate programminglanguage, such as JavaScript or VisualBasic.

Once a primary Study 1105 has been selected for display, the primarycondition of each rule is evaluated. Those rules that match, i.e.,evaluate to “true” for the given primary Study 1105, will then beapplied to all other studies that are available for the same patient.Those other studies for which the secondary condition matches will beadded to the list of studies to be displayed.

The following rule illustrates the concept. This rule will automaticallyload prior Chest X-Rays or prior Chest CT if the primary Study 1105 is aChest X-RAY.

Study Selection Rule 1:

-   -   IF (Primary.Dicom.BodyPartExamined=“CHEST” and        Primary.Dicom.Modality=“CR”)    -   THEN SELECT other studies for loading WHERE (Other.Dicom.        BodyPartExamined=“CHEST” and (Other.Dicom.Modality=“CR” or        Other.Dicom.Modality=“CT”)).

The rule is expressed in pseudo-code with the primary conditionspecified in the IF-clause and the secondary condition expressed in theSELECT-clause.

Study Selection Rule 2A:

-   -   IF (Primary.Dicom.BodyPartExamined=“SPINE” and        Primary.Dicom.Modality=“MR”)    -   THEN SELECT other studies for loading WHERE (Other.Dicom.        BodyPartExamined=“SPINE” and (Other.Dicom.Modality=“MR” AND        INTERSECTION(Primary.Vertebrae,Other.Vertebrae) NOT EMPTY).

Study Selection Rule 2B:

-   -   IF (Primary.Dicom.BodyPartExamined=“SPINE” and        Primary.Dicom.Modality=“CT”)    -   THEN SELECT other studies for loading    -   WHERE (Other.Dicom. BodyPartExamined=“SPINE” and        Other.Dicom.Modality=“CT” AND        INTERSECTION(Spinous.process,Cobb.angle) NOT EMPTY).

Study Selection Rule 3A:

-   -   IF (Primary.Dicom.BodyPartExamined=“CHEST” and        Primary.Dicom.Modality=“MR”)    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.BodyPartExamined=“CHEST” and        Other.Dicom.Modality=“MR” AND        INTERSECTION(Spinous.process,Thyroid) NOT EMPTY).

Study Selection Rule 3B:

-   -   IF (Primary.Dicom.BodyPartExamined=“CHEST” and        Primary.Dicom.Modality=“CT”)    -   THEN SELECT other studies for loading WHERE (Other.Dicom.        BodyPartExamined=“CHEST” and Other.Dicom.Modality=“CT” AND        INTERSECTION(Spinous.process,Thyroid) NOT EMPTY).

Study Selection Rule 4A:

-   -   IF (Primary.Dicom.BodyPartExamined=“BREAST” and        Primary.Dicom.Modality=“MR”)    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.BodyPartExamined=“BREAST” and        Other.Dicom.Modality=“MR” AND INTERSECTION(Implant,Tumor) NOT        EMPTY).

Study Selection Rule 4B:

-   -   IF (Primary.Dicom.BodyPartExamined=“BREAST” and        Primary.Dicom.Modality=“CT”)    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.BodyPartExamined=“BREAST” and        Other.Dicom.Modality=“CT” AND INTERSECTION(Implant,Tumor) NOT        EMPTY).

Study Selection Rule 5A:

-   -   IF (Primary.Dicom.BodyPartExamined=“BODY_PART” and        Primary.Dicom.Modality=“MR”)    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.BodyPartExamined=“BODY_PART” and        Other.Dicom.Modality=“MR” AND        INTERSECTION(Parameter1,Parameter2) NOT EMPTY).

Study Selection Rule 5B:

-   -   IF (Primary.Dicom.BodyPartExamined=“BODY_PART” and        Primary.Dicom.Modality=“CT”)    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.BodyPartExamined=“BODY_PART” and        Other.Dicom.Modality=“CT” AND        INTERSECTION(Parameter3,Parameter4) NOT EMPTY).

Where Parameter3 can be but need not be equal to Parameter1 andParameter4 can be but need not be equal to Parameter2.

Study Selection Rule 5C:

-   -   IF (Primary.Dicom.BodyPartExamined=“BODY_PART” and        Primary.Dicom.Modality=“MG”)    -   THEN SELECT other studies for loading WHERE (Other.Dicom.        BodyPartExamined=“BODY_PART” and (Other.Dicom.Modality=“MG” AND        INTERSECTION(Parameter5,Parameter6) NOT EMPTY).

Where Parameter5 can be but need not be equal to Parameter1 and/orParameter3, and Parameter6 can be but need not be equal to Parameter2and/or Parameter4.

In this example if Parameter5 is Primary.Vertebrae and Parameter6 isOther.Vertebrae the Parameter5 and Parameter6 denote the set ofvertebrae in the primary and the other study respectively, andINTERSECTION( . . . ) NOT EMPTY selects only those prior studies forcomparison that actually show at least parts of the same anatomy. Thisway the rule would not select a cervical spine scan for comparison whenthe current study is a lumbar spine, but it would select a priorthoracic spine scan for comparison, if that scan did have an overlapwith the current scan of the lumbar spine.

Study Selection Rules: Normalization of DICOM Parameters

In an embodiment of the present invention, the rules can normalize DICOMparameters. As described above, a Study Selection Rule can containarbitrary DICOM parameters. The DICOM standard specifies if a particularparameter is defined on a patient, Study, series, or image level. Forexample, a Study-level parameter should have the same value in allimages of a Study, while a series-level parameter should have the samevalue in all images of a series. There are two problems related toassuming that this statement is always the case. Firstly, although aStudy-level tag should have the same value for all images of a Studythis is not always true. Secondly, some parameters are defined on aseries- or image-level (e.g. modality is a series-level parameter) andtherefore can be unavailable. In both cases it can be unclear what valueis to be used when evaluating the rule. The invention described hereprovides different solutions to this problem.

In an embodiment of the present invention, a first approach is to choosea reference image and to read the value of a particular DICOM parameterfrom the reference image. The reference image can be: (i) the image thatwas inserted into the system first, (ii) the image with the oldest imagecontent date, (iii) the image that was inserted into the system last, or(iv) the image with the earliest image content date. The choice of whichimage is to be chosen as the reference image can be configured for eachparameter separately.

In an embodiment of the present invention, a second approach is to onlyallow original images to be chosen as the reference image. Non-viewableDICOM objects like structured reports, key objects, or presentationstates are disregarded, as well as derived images such as secondarycapture images or reformatted images.

In an embodiment of the present invention, a third approach is toprovide a list of all distinct values that a particular DICOM parameterhas in the images of a Study. In a Study Selection Rule one can thencheck if that list contains a particular value. The example above canthen read as follows:

Study Selection Rule 6:

-   -   IF (Primary.Dicom.BodyPartExamined=“CHEST” and        Primary.DicomList.Modality contains “CR”)    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.BodyPartExamined=“CHEST” and        (Other.DicomList.Modality contains “CR” or        Other.DicomList.Modality contains “CT”)).

Study Selection Rules: Abstract Tags

In an embodiment of the present invention, the Study Selection Rules1115 contain other derived parameters such as Abstract Tags and/or ImageContent Based Parameters that characterize a Study in addition to orinstead of DICOM parameters. Abstract tags that are useful within StudySelection Rules 1115 include the following:

-   -   (i) RelativeStudyAge indicates relative age of Study in days        compared to primary Study 1105.    -   (ii) PriorIndex indicates an index that enumerates all other        studies from youngest to oldest.    -   (iii) NumImages indicates number of images in Study.    -   (iv) NumSeries indicated number of image series in Study.    -   (v) Num3DVolumes indicates number of 3D volumes in Study.    -   (vi) Num4DSequences indicates number of 4D sequences in Study        (e.g. Cardiac CT).    -   (vii) HasReport indicates a flag that indicates if a report is        available for a Study.    -   (viii) HasThinSliceVolumes indicates whether the study has at        least one set of images that form a true 3D volume, i.e. a        sufficiently large number of equidistant slices (the exact        number can be user configurable, e.g. 30 would be a common        choice) and a sufficiently small spacing between two consecutive        slices to guarantee an isotropic (or close to isotropic) (again,        this parameter can be user defined, values between 1 mm and 3 mm        are common thresholds for CT and MR examinations).

For example, a rule that applies to a Mammogram Study and that selectsat maximum three prior Mammogram studies no older than five years canread as follows.

Study Selection Rule 7:

-   -   IF (Primary.Dicom.Modality=“MG”    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.Modality=“MG” and Other.Abstract.Priorindex<=3 and        Other.Abstract.RelativeStudyAge<5*365).

Protocol Selection Rules 1135

In an embodiment of the present invention, once the studies to bedisplayed are determined as described above, a suitable display protocolcan be selected. This is done using matching rules. Each matching ruleconsists of conditions that are applied to the primary and other studiesto be loaded. Like in Study Selection Rules 1115, protocol selectionrules may contain DICOM parameters (either taken from a reference imageor provided as a list of distinct values gathered from all images of astudy), as well as Abstract Tags and/or Image Content Based Parametersand user-defined functions. Each matching rule has a score and anassociated display protocol.

In an embodiment of the present invention, all matching rules areevaluated and the display protocol of the matching rule that evaluatesto true can be selected. If multiple matching rules evaluate to true,the one with the highest score can be selected.

The following example rule illustrates a matching rule that can applyfor PET/CT studies of the abdomen to select a protocol named“StandardPetCTProtocol1” with a score of 10.

Protocol Selection Rule 1:

-   -   IF (Primary.Dicom.BodyPartExamined=“ABDOMEN” and        Primary.Dicom.Modality=“CT” and Exists(Other1) and        Other1.Dicom.Modality=“PET”    -   THEN SELECT “StandardPetCTProtocol1” with score=10.

In an embodiment of the present invention, the rule is expressed inpseudo-code with the matching condition specified in the IF-clause andthe chosen protocol specified by the SELECT.

Image Set Rules 1150

In an embodiment of the present invention, once a display protocol isselected, further rules defined within the protocol are evaluated. Thenext step comprises creation of so-called image sets. An image setconsists of images that are logically grouped together. Usually, animage set is represented by a single preview icon in the application. Itis an image set that is loaded into a viewer or tiled viewer. Note thatDICOM series also represent a logical grouping of images. However, oftenDICOM series are not well suited for hanging of images and viewing. Forexample, in Mammography a single DICOM series may contain images of bothleft and right breast, in MRI it may contain both T1 and T2 images, orin CT it may contain both a localizer image and a 3D image stack. In allthese cases the DICOM series can be split into different logical imagesets. On the other hand, multiple DICOM series may represent the phasesof a single 4D cardiac data set. In this case all those series can bejoined into a single logical image set.

In an embodiment, the logical image set is a 4D cardiac image set. In anembodiment, the individual images of the 4D cardiac image set are sortedin time to represent the temporal ordering of the cardiac cycle. In anembodiment, CNN is applied to the images of the 4D cardiac image inorder to determine the presence of pathological indicators, e.g.,myocardial infarction. In embodiments, evidence of pathologicalindicator is memorialized in one or more Image Content Based Parameters.

Thus the creation of image sets based on rules is a key component of therule-based display system, specifically for the more advanced renderingtechniques. For example, the rules-based display system is used tocreate image sets that are very similar to the rules described above inStudy Selection Rules 1115 and Protocol Selection Rules 1135 sections. Arule is a Boolean expression that can contain DICOM parameters, abstracttags, Image Content Based Parameters, or used-defined functions that arebased on the DICOM parameters, abstract tags, Image Content BasedParameters. Image set rules however, are applied to all images of astudy that was selected for loading (and not to the study itself).Image-level parameters thus represent no problem and do not need to benormalized or otherwise treated specially. All images that match animage-set rule are grouped into a respective image set.

In an embodiment of the present invention, the following rule (expressedin pseudo-code) collects all images of a current CT study.

Image Set Rule 1:

-   -   IF (Dicom.Modality=“CT” and Abstract.Priorindex=0)    -   THEN CREATE image set with ID 1.

In an embodiment of the present invention, the resulting image sets canbe assigned IDs or names that allow for referencing the image sets laterin layout and display set rules. In the above example an image set withID 1 was defined. If no image matches an image set rule, no suchcorresponding image set will be created.

Image Set Rules: Sorting

In an embodiment of the present invention, the order of images within animage set is an important aspect. It determines how images are shownwhen the user browses through the image set or how images aredistributed into the tiles of a tiled viewer. In one embodiment of thepresent invention, in order to specify image sorting, the image setrules can contain an ordered list of sorting criteria. All images thatare matched by a rule are sorted according to those criteria.

For example, the following rule collects all images of a current CTstudy and sorts them according to DICOM series number at first and DICOMinstance/image number at second.

Image Set Rule 2:

-   -   IF (Dicom.Modality=“CT” and Abstract.Priorindex=0)    -   THEN CREATE image set with ID 1        -   SORTED BY Dicom.SeriesNumber ORDER:=ascending        -   SORTED BY Dicom.InstanceNumber ORDER:=ascending.

Image Set Rules: Splitting

In an embodiment of the present invention, sorting criteria can beextended by a split flag. With the split flag it is possible to createmultiple image sets from a single image set rule. When the value of asorting criterion with split flag set to true changes, sub-sequentimages are automatically inserted into a new image set. The resultingimage sets are automatically enumerated by a sub-level ID.

For example, the following rule essentially creates image sets thatcorrespond to DICOM series, because all images with different seriesnumber will be split into different sets.

Image Set Rule 3:

-   -   IF (Dicom.Modality=“CT” and Abstract.Priorindex=0)    -   THEN CREATE image set with ID 1.x        -   SORTED BY Dicom.SeriesNumber ORDER:=ascending SPLIT:=true        -   SORTED BY Dicom.InstanceNumber ORDER:=ascending            SPLIT:=false.

In applications where a CT has been measured, it can happen that a studycontains both a soft-kernel series and a hard kernel series and bothseries have the same series number. In order to separate the images intodifferent image sets the above rule can be extended by the following:

Image Set Rule 4:

-   -   IF (Dicom.Modality=“CT” and Abstract.Priorindex=0)    -   THEN CREATE image set with ID 1.x        -   SORTED BY Condition.CTSoftTisseKernel SPLIT:=true        -   SORTED BY Dicom.SeriesNumber ORDER:=ascending SPLIT:=true        -   SORTED BY Dicom.InstanceNumber ORDER:=ascending            SPLIT:=false.

Here, Condition.CTSoftTissueKernel denotes a user-defined Booleancondition that tests whether an image has a CT soft-tissue kernel. Theactual implementation of this condition can for example evaluate themanufacturer (which is encoded in a DICOM parameter). Depending on itsvalue the rule can evaluate further parameters to find out if an imagewas reconstructed using a soft-tissue kernel or not. Since this Booleancondition was used as a sorting criterion with the split flag set totrue, all non-soft-kernel images can be put into an image set with ID1.1 and all soft-kernel images can be put into an image set with ID 1.2(unless the image set is further split and IDs like 1.3 or 1.4 arecreated).

Image Set Rules: More Abstract Tags

In an embodiment of the present invention, additional Abstract Tagsand/or Image Content Based Parameters are used in image set rules. Oneexample is a tag that identifies whether an image has already been putinto an image set. In principle, a single image can be put into multipleimage sets, but sometimes this should be avoided. This can be achievedby evaluating image set rules in a pre-defined order and introducing anabstract tag AlreadyReferenced.

For example, in CT study that has a localizer image and a 3D image stackboth stored in one DICOM series, one may want to create an image set,one for the localizer and one for the 3D image stack. Accordingly, theimage set rules are defined as follows.

Image Set Rule 5 (Localizer):

-   -   IF (Dicom.Modality=“CT” and Condition.IsLocalizer=true)    -   THEN CREATE image set with ID 1        -   SORTED BY Dicom.SeriesNumber ORDER:=ascending SPLIT:=true        -   SORTED BY Dicom.InstanceNumber ORDER:=ascending            SPLIT:=false.

Image Set Rule 6 (Images):

-   -   IF (Dicom.Modality=“CT” and Abstract.AlreadyReferenced=false)    -   THEN CREATE image set with ID 2        -   SORTED BY Dicom.SeriesNumber ORDER:=ascending SPLIT:=true        -   SORTED BY Dicom.InstanceNumber ORDER:=ascending            SPLIT:=false.

Here Condition.IsLocalizer is a user-defined condition that returns trueif an image is a localizer image, and false otherwise. In an embodimentof the present invention, Rule 1 is applied first. Therefore thelocalizer image is put into a separate image set with ID 1. Next rule 2is applied. This rule can match for all CT images including thelocalizer image. However, because AlreadyReferenced=false is specified,the localizer image is skipped and not placed into image set 2.

In an embodiment of the present invention, the creation of the imagesets based on rules is a key component of the efficient rules baseddisplay, specifically for the more advanced rendering techniques. Forexample rules can be used to identify sets of 2D images that togetherform a 3D volume.

Viewer Assignment Rules

In another embodiment of the present invention, a display protocoldefines multiple viewers, each with one or more tiles, i.e., viewports.To each viewer one or more image sets can be assigned based on ViewerAssignment Rules that are similar to the protocol section rulesdescribed herein. Viewer Assignment Rules are defined in the displayprotocol. The rules determine which image set shall be initially shownin a viewer. In case multiple image sets are assigned to a viewer, theone with the highest score is chosen. Afterwards users may cycle quicklythrough the remaining image sets using dedicated tools (Previous/NextImage Set), or pick another image set from a special image set menu.

Like the other rule types Viewer Assignment Rules contain Booleanexpressions of DICOM parameters, Abstract Tags, Image Content BasedParameters, or user-defined conditions based on DICOM parameters, ImageContent Based Parameters, or abstract tags. In many cases it issufficient to specify the image sets to be assigned to a viewer by theirimage set ID instead of evaluating the underlying DICOM parameters,Image Content Based Parameters and/or Abstract Tags again. Therefore,the image set ID is simply set as a separate abstract tag. In thefollowing example the two rules load image sets with the IDs 1 and 2into a viewer, but assign ID 1 a higher score so that this image set isinitially visible (provided such an image set exists).

Viewer Assignment Rule 1:

-   -   IF (EXISTS ImageSet[1])    -   THEN Viewport[0].AddlmageSet(ID=1, score=10).

Viewer Assignment Rule 2:

-   -   IF (EXISTS ImageSet[2])    -   THEN Viewport[0].AddImageSet(ID=2, score=5).

In an embodiment of the present invention, viewer assignment rules areapplied to image sets. Thus there is a possible conflict regardingambiguous image-level and series-level tags. This conflict is resolvedin the same way as described herein in the Normalization of DICOMParameters section. This means that values of DICOM parameters, ImageContent Based Parameters, but also Abstract Tags, are automaticallytaken from some reference image. Alternatively, for all DICOM parametersa list of distinct values occurring in all images of the image set canbe used in an assignment rule.

Style Rules

In one embodiment of the present invention, there is a final set ofrules that specify the rendering style and other rendering parameters tobe used when showing a particular image set. For example, for a CTAngiogram study, often a volume rendering style display (VRT) isdesired, whereas for a study looking for lung nodules a maximumintensity projection (MIP) of 20 mm slabs may be desired. Style rules,that can be user specific, allow driving that automatically. The rulescan use the same parameters as discussed above, as well as the existenceor absence of certain image sets.

In one embodiment of the present invention, the system uses a global,ordered list of style rules that is evaluated independently for eachviewer and each image set loaded into a viewer. An abstract tagDisplaySetID is provided that allows formulating style rules for aspecific viewer or group of viewers.

Parameters driven by Style Rules include the following:

-   -   Rendering style (can be 2D, oblique, curved, MIP slab, 3D MIP,        VRT, shaded VRT, etc.);    -   Image alignment (left, right, top, bottom, centered);    -   Inverse display (black on white versus white on black);    -   Colormap or transfer function;    -   Window/level (data window);    -   Slice thickness;    -   Zoom factor;    -   Camera position and orientation; and    -   Labels/OverlayDisplay of labels, annotations and other overlay        elements.

The following is an example of a style rule that activates inverse 3DMIP rendering in all viewers with a DisplaySetID between 101 and 104,provided a PET data set is loaded into those viewers (modality PT, i.e.,positron emission tomography). Also, an automatic window/level settingis used that is computed from the histogram of the image set (the 2%lowest values are all mapped to white, and the 2% highest values are allmapped to black):

Style Rule 1:

-   -   IF (Abstract.DisplaySetID>100 and        -   Abstract.DisplaySetID<105 and        -   Dicom.Modality=“PT”)    -   THEN SET        -   RenderingStyle:=“3D MIP”        -   Inverse:=true        -   DataWindow:=“2% 98%”

The following is another example of a different style rule that alwayscauses the image set with image set ID 200 to be displayed in MPR modeusing 20 mm thick slices, with a window/level as specified in the DICOMparameters, and with a zoom factor so that the whole viewer window isfilled out. The rule is as follows.

Style Rule 2:

-   -   IF (Abstract.ImageSetID=200)    -   THEN SET        -   RenderingStyle:=“MPR”        -   SliceThickness:=“20”        -   DataWindow:=“DICOM1”        -   ZoomFactor:=“FitToWindow”

Summary of Rule Types

Table I summarizes all types of rules that are applied in the rule-basedisplay system:

TABLE I Normalized Rule Type Applies to Parameters Defined where StudySelection Rule Studies yes globally Protocol Selection Rule Studies yesglobally Image Set Rule Images not required protocol Viewer AssignmentRule Image Sets yes globally, protocol Style Rule Image Sets yesglobally, protocol

Described above are methods and systems for implementing a rule derivedbasis to display image sets. The foregoing description of embodiments ofthe methods, systems, and components of the present invention has beenprovided for the purposes of illustration and description. It is notintended to be exhaustive or to limit the invention to the precise formsdisclosed. Many modifications and variations will be apparent to one ofordinary skill in the relevant arts. For example, steps performed in theembodiments of the invention disclosed can be performed in alternateorders, certain steps can be omitted, and additional steps can be added.The embodiments were chosen and described in order to best explain theprinciples of the invention and its practical application, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with various modifications that are suited tothe particular used contemplated. Other embodiments are possible and arecovered by the invention. Such embodiments will be apparent to personsskilled in the relevant art(s) based on the teachings contained herein.The breadth and scope of the present invention should not be limited byany of the above-described exemplary embodiments, but should be definedonly in accordance with the following claims and their equivalents.

Example Shown in FIG. 2

An example of how these aspects can be combined is shown in FIG. 2. Inthe example the user has selected a CT examination of the abdomen. Thefollowing rules have been used to determine that a recent X-Ray of thechest is relevant and shall be displayed as well:

-   -   IF (Primary.Dicom.BodyPartExamined=“ABDOMEN” and        Primary.Dicom.Modality=“CT”)    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.BodyPartExamined=“ABDOMEN” OR        Other.Dicom.BodyPartExamined=“CHEST”) and        (Other.Dicom.Modality=“CR” or Other.Dicom.Modality=“CT”) AND        Other.RelativeStudyAge<“90 days”

From this rule, a hanging protocol can be selected. In the example theprotocol selection rules determine that the CT study is a thin slice CTstudy (i.e. that it has image series that form a 3D volume withsufficient resolution in all directions to display volume rendering ornon-axial slices in a meaningful way). Furthermore the example ruledetermines that this is a study with enhanced vasculature, by lookingfor the key words “contrast” or “angio” in the study description.

The display protocol selection rule that applies here and select theprotocol CTThinSliceVesselWithPrior can read:

-   -   IF (Primary.Dicom.BodyPartExamined=“ABDOMEN” and        Primary.Dicom.Modality=“CT” and        Primary.Abstract.HasThinSliceVolumes and        (Primary.Dicom.StudyDescription containsAnyOf “contrast, angio”        and exists Other 1 THEN SELECT “CTThinSliceVesselWithPrior” with        score=10.

From this image sets are generated using Image Set Rules:

-   -   IF (Dicom.Modality=“CT” and Abstract.Priorindex=0 and        Condition.IsPartOfThinSliceVolume and        Condition.CTSoftTisseKernel)    -   THEN CREATE image set with ID 1.x        -   SORTED BY Abstract.NumberOfSlicesInVolume ORDER:=descending            SPLIT:=true        -   SORTED BY Dicom.SeriesNumber ORDER:=ascending SPLIT:=true        -   SORTED BY Dicom.Abstract.VolumeIndex ORDER:=ascending            SPLIT:=true        -   SORTED BY Dicom.Abstract.SlicePosition ORDER:=ascending            SPLIT:=false.

This rule will actually form sets from images that contain images thatare part of a ThinSliceVolume and that have been reconstructed with a“soft tissue” kernel. Given the protocol selection rule has specificallymatched for just CT studies, the conditions Dicom.Modality=“CT” andAbstract.Priorindex=0 are actually redundant, but could be useful if adifferent selection rule was used.

The images will first be sorted by the size of the volume of which theyare part (Abstract.NumberOfSlicesInVolume) , then by DICOM series. Thesplit parameter in this case will ensure that an image set containsimages from on series only. A DICOM series can sometimes containmultiple non-consecutive volumes. The abstract tag VolumeIndex will thenindicate for each image, which of those volumes it is part of. If aseries contains only one volume, then this will be “1” for all images inthe series. The split=true in this part of the rule would result in aseparate image set for each of those volumes. Finally, within eachvolume, the images are ordered by slice position, but not split. Thisway we end up with one image set for each soft kernel thin slice volume,the largest volume being the first image set (ID 1.1). This ID will beused further in subsequent rules.

The rule to form an image set from any CR prior study in this example ismuch simpler:

-   -   IF (Dicom.Modality=“CR” and Abstract.Priorindex=1)    -   THEN CREATE image set with ID 10        -   SORTED BY Dicom.SeriesNumber ORDER:=ascending SPLIT:=false        -   SORTED BY Dicom.InstanceNumber ORDER:=ascending            SPLIT:=false.

This creates one image set with ID=10 containing all images for thefirst prior study, if that is a CR.

In practice, additional rules, such as Image Set Rule 5 and 6 (seeabove) will be used to collect the remaining images of the primary Study1105. The remaining images are not shown in the layout depicted in theexample FIG. 2.

The Display Protocol 1145 contains multiple layouts. The one shown inFIG. 2 is defined as follows:

DEFINE Layout { ID=“Layout5”; NAME=”+PlainFilm” Viewports { { ID=50,Geometry=”(0,0)−(0.25,0.5)”}, { ID=51, Geometry=”(0.25,0)−(0.5,0.5)”}, {ID=52, Geometry=”(0,0.5)−(0.25,1)”}, { ID=53, Geometry=”(0.25,0.5)−(0.5,0.5)”}, { ID=54, Geometry=”(0.5,0)−(1,1)”, Style=”2D”} } }

In this example the geometry is defined in a coordinate system havingthe origin in the upper left corner of the screen with the x axispointing to the right and the y axis pointing down. Please note howparameters of the viewers can be set in the layout definition.Parameters can also be set or overridden in the assignment and stylerules, as will be explained next.

In this example, viewer assignment and style rules are as follows:

IF ImageSetExists (1.1) and ImageSetExists(10) THEN SHOW_LAYOUT Layout5WITH Viewport[0].AddImageSet(1.1) Viewport [0].Style=”VRT(diffuse)”Viewport [0].Colormap=”CTAngioSoftTissue” Viewport[1,2,3].AddImageSet(1.1) Viewport [1,2,3].Style=”MPR” Viewport[1,2,3].DataWindow=”DICOM1” Viewport [1].oriantation=”axial” Viewport[2].oriantation=”sagittal” Viewport [3].oriantation=”coronal” Viewport[4].AddImageSet(10) IF (ImageSet[10].Dicom.Columns > 1024) THENViewport[4].Zoom=”FitToWindow” ELSE Viewport[4].Zoom=”1:1”

In this particular example, the rule to select the layout is rathersimple: It is shown if the two image sets used exist. This is becausethe criteria to construct these images sets have been rather specific.As will be appreciated, the proposed system gives this type offlexibility.

ASPECTS OF THE INVENTION

Some aspects of this invention include methods of displaying one or moreSets of Images comprising the steps of:

-   -   a. selecting a primary Study;    -   b. selecting one or more Study Selection Parameters based on the        primary Study;    -   c. selecting one or more Study Selection Rules based on the one        or more Study Selection Parameters;    -   d. selecting one or more Sets of Images from a plurality of        images based on the one or more Study Selection Rules;    -   e. selecting one or more Display Protocol Selection Parameters        based on the one or more Sets of Images selected;    -   f. selecting one or more Display Protocol Selection Rules based        on the one or more Display Protocol Selection Parameters;    -   g. selecting one or more Display Parameters using the one or        more Display Protocol Selection Rules; and    -   h. displaying the one or more Sets of Images according to the        Display Parameters.

Additional aspects include methods one or more Display Parameter areselected from the group consisting of Image Set Selection Parameters andView and Viewport Selection Parameters.

Further aspects include methods where the one or more Display Parametersare selected from the group consisting of Image Set Selection Rules,View and Viewport Selection Rules, and Display Protocol Selection Rules.

Yet further aspects include methods where the step of identifying one ormore Image Set Selection Rules is based on the one or more Image SetSelection Parameters.

Still further aspects include methods where the step of selecting one ormore Viewpoint Selection Rules is based on one or more View and ViewportSelection Parameters.

Other aspects include methods where the step of displaying the one ormore Sets of Images is based on one or more Display Protocol SelectionRules, one or more Image Set Selection Rules, and one or more View andViewport Selection Rules.

Still other aspects include methods where one or more of the StudySelection Parameters are selected from the group consisting of DICOMparameters, Image Content Based Parameters, and Abstract Tags.

Other aspects include methods where one or more of the Display ProtocolSelection Parameters are selected from the group consisting of DICOMparameters, Image Content Based Parameters, and Abstract Tags.

Additional aspects include methods where one or more of the Image SetSelection Parameters are selected from the group consisting of DICOMparameters, Image Content Based Parameters, and Abstract Tags.

Further aspects include methods where one or more of the View andViewport Selection Parameters are selected from the group consisting ofDICOM parameters, Image Content Based Parameters, and Abstract Tags.

More aspects include methods where one or more Study SelectionParameters are derived from a single reference image.

Still more aspects include methods where one or more Study SelectionParameters are derived from a single reference image DICOM Parameters.

Yet other aspects include methods where one or more Display ProtocolSelection Parameters are derived using a list of all values of a DICOMparameter occurring in any of the one or more Sets of Images.

Alternative aspects include methods where the one or more View andViewport Selection Rules contain protocols for one or more Viewportsdisplaying images as 2D.

Other alternative aspects include methods where the one or more View andViewport Selection Rules contain protocols for one or more Viewportsdisplaying images in a 3D rendering mode.

Further alternative aspects include methods where one or more StudySelection Parameters include one or more Abstract Tags selected from thegroup consisting of RelativeStudyAge, PriorIndex. NumImages, NumSeries,Num3DVolumes, Num4DSequences and HasReport.

In other aspects, this invention includes methods where one or more Viewand Viewport Selection Rules include one or more Abstract Tags selectedfrom the group consisting of Image Sets to be displayed, RenderingStyle, Additional image sets for image fusion, Image Alignment,Colormap/Transfer Function, Slice Thickness, Zoom Factor, Cameraposition, Camera orientation and Labels/Overlay elements or one or moreImage Content Based Parameters.

In still other aspects, this invention includes methods furthercomprising the steps of:

-   -   receiving one or more Sets of Images based on the Study        Selection Rules;    -   selecting one or more Image Set Selection Parameters;    -   selecting one or more Image Set Selection Rules based on the one        or more Image Set Selection Parameters; and    -   displaying the one or more Sets of Images based on the Display        Protocol Selection Rules and the Image Set Selection Rules.

In another aspect, this invention includes methods of displaying one ormore Sets of Images comprising the steps of:

-   -   selecting one or more Study Selection Parameters;    -   selecting or more Study Selection Rules based on the one or more        Study Selection Parameters;    -   receiving one or more Sets of Images based on the Study        Selection Rules;    -   selecting one or more Display Protocol Selection Parameters        based on the one or more Sets of Images selected;    -   selecting one or more Display Protocol Selection Rules based on        the one or more Display Protocol Selection Parameters; and    -   displaying the one or more Sets of Images based on the Display        Protocol Selection Rules.

Another aspect of this invention includes methods of displaying imagescomprising the steps of:

-   -   a. selecting one or more Study Selection Parameters;    -   b. selecting Study Selection Rules based on the one or more        Study Selection Parameters;    -   c. receiving one or more images based on the Study Selection        Rules;    -   d. selecting one or more Display Protocol Selection Parameters        based on the one or more images selected;    -   e. selecting Display Protocol Selection Rules based on the one        or more Display Protocol Selection Parameters;    -   f. selecting one or more Image Set Selection Parameters;    -   g. selecting Image Set Selection Rules based on the one or more        Image Set Selection Parameters;    -   h. selecting one or more View and Viewport Selection Parameters;    -   i. selecting View and Viewport Selection Rules based on the one        or more View and Viewport Selection Parameters; and    -   j. displaying the one or more images based on the Display        Protocol Selection Rules, the Image Set Selection Rules and the        View and Viewport Selection Rules.

Other aspects of the invention include methods where the Study SelectionRule is:

-   -   IF (Primary.Dicom.BodyPartExamined=“CHEST” and        Primary.Dicom.Modality=“CR”)    -   THEN SELECT other studies for loading WHERE (Other.Dicom.        BodyPartExamined=“CHEST” and (Other.Dicom.Modality=“CR” or        Other.Dicom.Modality=“CT”)).

In another aspect, this invention includes methods where the StudySelection Rule is:

-   -   IF (Primary.Dicom.BodyPartExamined=“CHEST” and        Primary.DicomList.Modality contains “CR”)    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.BodyPartExamined=“CHEST” and        (Other.DicomList.Modality contains “CR” or        Other.DicomList.Modality contains “CT”)).

In other aspects, this invention includes methods where the StudySelection Rule is:

-   -   IF (Primary.Dicom.Modality=“MG”    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.Modality=“MG” and Other.Abstract.Priorindex<=3 and        Other.Abstract.RelativeStudyAge<5*365).

In yet another aspect, this invention includes methods where theProtocol Selection Rule is:

-   -   IF (Primary.Dicom.BodyPartExamined=“ABDOMEN” and        Primary.Dicom.Modality=“CT” and Exists(Other1) and        Other1.Dicom.Modality=“PET”)    -   THEN SELECT “StandardPetCTProtocol1” with score=10.

In aspects of the invention, methods include an Image Set Rule:

-   -   IF (Dicom.Modality=“CT” and Abstract.Priorindex=0)    -   THEN CREATE image set with ID 1.

Additionally, other aspects include methods where the Image Set Rule is:

-   -   IF (Dicom.Modality=“CT” and Abstract.Priorindex=0)    -   THEN CREATE image set with ID 1        -   SORTED BY Dicom.SeriesNumber ORDER:=ascending        -   SORTED BY Dicom.InstanceNumber ORDER:=ascending.

Still other aspects include methods where the Image Set Rule is:

-   -   IF (Dicom.Modality=“CT” and Abstract.Priorindex=0)    -   THEN CREATE image set with ID 1.x        -   SORTED BY Dicom.SeriesNumber ORDER:=ascending SPLIT:=true        -   SORTED BY Dicom.InstanceNumber ORDER:=ascending            SPLIT:=false.

Moreover, other aspects include methods where the Image Set Rule is:

-   -   IF (Dicom.Modality=“CT” and Abstract.Priorindex=0)    -   THEN CREATE image set with ID 1.x        -   SORTED BY Condition.CTSoftTisseKernel SPLIT:=true        -   SORTED BY Dicom.SeriesNumber ORDER:=ascending SPLIT:=true        -   SORTED BY Dicom.InstanceNumber ORDER:=ascending            SPLIT:=false.

Yet other aspects include methods where the Image Set Rule (Localizer)is:

-   -   IF (Dicom.Modality=“CT” and Condition.IsLocalizer=true)    -   THEN CREATE image set with ID 1        -   SORTED BY Dicom.SeriesNumber ORDER:=ascending SPLIT:=true        -   SORTED BY Dicom.InstanceNumber ORDER:=ascending            SPLIT:=false.

Other aspects of the methods of this invention include an Image Set Rule(Images):

-   -   IF (Dicom.Modality=“CT” and Abstract.AlreadyReferenced=false)    -   THEN CREATE image set with ID 2        -   SORTED BY Dicom.SeriesNumber ORDER:=ascending SPLIT:=true        -   SORTED BY Dicom.InstanceNumber ORDER:=ascending            SPLIT:=false.

Yet other aspects of the methods of this invention include using ImageSet Rule (Images):

-   -   IF (Dicom.Modality=“CT” and Abstract.AlreadyReferenced=false)    -   THEN CREATE image set with ID 2        -   SORTED BY Dicom.SeriesNumber ORDER:=ascending SPLIT:=true        -   SORTED BY Dicom.InstanceNumber ORDER:=ascending            SPLIT:=false.

Additionally, other aspects include methods where the Display Parametersinclude Viewer Assignment Rule:

-   -   IF (Abstract.ImageSetID=1)    -   THEN SELECT image set with score=10.

Yet further aspects include methods where the Display Parameters includea Viewer Assignment Rule:

-   -   IF (Abstract.ImageSetID=2)    -   THEN SELECT image set with score=5.

Additional aspects include methods further comprising a ViewerAssignment Rule:

-   -   IF (Abstract.ImageSetID=2)    -   THEN SELECT image set with score=5.

Further Embodiments

Further embodiments contemplated herein include the following.

In an aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting as a ParameterA an anatomical characteristic in the primarystudy and a ParameterB as Modality in the primary study; (c) executingon a server digital data processor a render server program which appliesone or more Study Selection Rules to: (i) generate a list of a pluralityof secondary studies based on ParameterA and ParameterB; (ii) generatefrom the list of the plurality of secondary studies one or moreImageContentBased parameters using Convolutional Neural Networks(CNN);(iii) select from the list of the plurality of secondary studies afinal list based on the one or more ImageContentBased parameters; (d)executing on the server digital data processor the render server programwhich applies one or more Protocol Selection Rules to select a DisplayProtocol, where the one or more Protocol Selection Rules are based ontwo or more parameters selected from the group consisting of one or moreDICOM parameters from the primary Study, one or more Abstract Tags fromthe primary Study, one or more DICOM parameters from the plurality ofsecondary studies, one or more Abstract Tags from the plurality ofsecondary studies and one or more ImageContentBased parameters; and (e)displaying the primary study and one or more of the plurality ofsecondary studies selected from the list based on the Display Protocolselected in step (d).

In an embodiment, the one or more ImageContentBased parametersidentified are present in the primary study. In an embodiment, the oneor more ImageContentBased parameters are vertebrae.

In an embodiment, the vertebrae in the primary study are selected fromthe group consisting of L1, L2, L3, L4, and L5, and at least one of thevertebra in the secondary study is a vertebra present in the primarystudy. In an embodiment, the vertebrae in the primary study are selectedfrom the group consisting of C1, C2, C3, C4, C5, C6, and C7, and atleast one of the vertebra in the secondary study is a vertebra presentin the primary study. In an embodiment, the vertebrae in the primarystudy are selected from the group consisting of Th1, Th2, Th3, Th4, Th5,Th6, Th7, Th8, Th9, Th10, Th11, and Th12, and at least one of thevertebra in the secondary study is a vertebra present in the primarystudy. In an embodiment, the CNN is pretrained with the plurality ofstudies. In an embodiment, the CNN is pretrained with a first pluralityof studies where the first plurality of studies is selected based on theanatomical characteristic in the primary study. In an embodiment, theCNN is pretrained with a first plurality of studies where the firstplurality of studies is selected based on one or more of theImageContentBased parameters.

In an embodiment, the method further includes the CNN selecting based onpsueudo code:

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“SPINE” and    -   Primary.Dicom.Modality=“MR”)

-   THEN SELECT other studies for loading

-   WHERE (Other.Dicom.AnatomicalCharacteristic=“SPINE” and    Other.Dicom.Modality=“MR” AND    INTERSECTION(Primary.Vertebrae,Other.Vertebrae) NOT EMPTY).

In an embodiment, the one or more Study Selection Rules restrict thefinal list to studies of the patient.

In another aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting a ParameterA from the group consisting of an anatomicalcharacteristic and a disease based characteristic in the primary study;(c) executing on a server digital data processor a render server programwhich applies one or more Study Selection Rules to: (i) generate a listof a plurality of secondary studies based on ParameterA; (ii) generatefrom the list of the plurality of secondary studies one or moreImageContentBased parameters using Convolutional Neural Networks (CNN);(iii) select from the list of the plurality of secondary studies a finallist based on the one or more ImageContentBased parameters; (d)executing on the server digital data processor the render server programwhich applies one or more Protocol Selection Rules to select a DisplayProtocol, where the one or more Protocol Selection Rules are based ontwo or more parameters selected from the group consisting of one or moreDICOM parameters from the primary Study, one or more Abstract Tags fromthe primary Study, one or more DICOM parameters from the plurality ofsecondary studies, one or more Abstract Tags from the plurality ofsecondary studies and one or more ImageContentBased parameters; and (e)displaying the primary study and one or more of the plurality ofsecondary studies selected from the list based on the Display Protocolselected in step (d).

In an embodiment, the method further includes selecting a ParameterB,where the plurality of secondary studies exclude one or more based onParameterB. In an embodiment, ParameterB is Modality. In an embodiment,ParameterB is selected from the group consisting of Computer Tomography(CT), then the Modality in the two or more secondary studies inserted isselected from the group consisting of Computed Radiography (CR), DigitalRadiography (DX), Mammography (MG), Magnetic Resonance (MR), OpthalmicPhotography (OP), Positron Emission Tomography (PT), Radio Fluoroscopy(RF), and X-Ray Angiography (XA). In an embodiment, ParameterB in theprimary study is equal to the ParameterB in the secondary study.

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   SELECT other studies for loading, WHERE        INTERSECTION(Primary.Dicom.Modality,Other.Dicom.Modality) NOT        EMPTY.

In an embodiment, ParameterA is BodyPartExamined. In an embodiment,ParameterA is selected from the group consisting of SPINE, CHEST,ABDOMEN, BREAST, SHOULDER, TRAPEZIUS, ARM, ELBOW, WRIST, FINGER, PELVIS,HIP, FIBULA, KNEE, TIBULA, ANKLE, FOOT, NECK, HEAD, TEMPOROMANDIBULARJUNCTION, FACE, BRAIN, DENTITION, SINUS, ADRENALS, RETINA, PITUITARY,and PROSTATE.

In an embodiment, the one or more Study Selection Rules restrict thefinal list to studies of the patient. In an embodiment, the methodfurther includes where a Study Selection Rule of the one or more StudySelection Rules in step (c) requires

-   -   SELECT other studies for loading, WHERE        INTERSECTION(Primary.Dicom.Modality,Other.Dicom.Modality) NOT        EMPTY.

In an embodiment, the ImageContentBased parameter is stored as a privateDICOM tag.

In another aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting as a ParameterA a disease-based characteristic in the primarystudy and selecting as a ParameterB a Modality in the primary study; (c)executing on a server digital data processor a render server programwhich applies one or more Study Selection Rules to: (i) generate a listof a plurality of secondary studies based on ParameterA and ParamaterB;(ii) generate from the list of the plurality of secondary studies one ormore ImageContentBased parameters using Convolutional Neural Networks(CNN); (iii) select from the list of the plurality of secondary studiesa final list based on the one or more ImageContentBased parameters; (d)executing on the server digital data processor the render server programwhich applies one or more Protocol Selection Rules to select a DisplayProtocol, where the one or more Protocol Selection Rules are based ontwo or more parameters selected from the group consisting of one or moreDICOM parameters from the primary Study, one or more Abstract Tags fromthe primary Study, one or more DICOM parameters from the plurality ofsecondary studies, one or more Abstract Tags from the plurality ofsecondary studies and one or more ImageContentBased parameters; and (e)displaying the primary study and one or more of the plurality ofsecondary studies selected from the list based on the Display Protocolselected in step (d).

In an embodiment, the one or more ImageContentBased parametersidentified are present in the primary study. In an embodiment, the oneor more ImageContentBased parameters are vertebrae. In an embodiment,the vertebrae in the primary study are selected from the groupconsisting of L1, L2, L3, L4, and L5, and at least one of the vertebraein the secondary study is a vertebra present in the primary study. In anembodiment, the vertebrae in the primary study are selected from thegroup consisting of C1, C2, C3, C4, C5, C6, and C7, and at least one ofthe vertebrae in the secondary study is a vertebra present in theprimary study. In an embodiment, the vertebrae in the primary study areselected from the group consisting of Th1, Th2, Th3, Th4, Th5, Th6, Th7,Th8, Th9, Th10, Th11, and Th12, and at least one of the vertebrae in thesecondary study is a vertebra present in the primary study. In anembodiment, the CNN is pretrained with the plurality of studies. In anembodiment, the CNN is pretrained with a first plurality of studieswhere the first plurality of studies is selected based on thedisease-based characteristic in the primary study. In an embodiment, theCNN selects one or more secondary studies which show the same anatomy.In an embodiment, the one or more Study Selection Rules restrict thefinal list to studies of the patient.

In another aspect there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting as a ParameterA an anatomical characteristic in the primarystudy and selecting as a ParameterB a Modality in the primary study; (c)executing on a server digital data processor a render server programwhich applies one or more Study Selection Rules to: (i) generate a listof a plurality of secondary studies based on ParameterA and ParamaterB;(ii) generate from the list of the plurality of secondary studies one ormore ImageContentBased parameters using Convolutional Neural Networks(CNN); (iii) select from the list of the plurality of secondary studiesa final list based on the one or more ImageContentBased parameters; (d)executing on the server digital data processor the render server programwhich applies one or more Protocol Selection Rules to select a DisplayProtocol, where the one or more Protocol Selection Rules are based ontwo or more parameters selected from the group consisting of one or moreDICOM parameters from the primary Study, one or more Abstract Tags fromthe primary Study, one or more DICOM parameters from the plurality ofsecondary studies, one or more Abstract Tags from the plurality ofsecondary studies and one or more ImageContentBased parameters; and (e)displaying the primary study and one or more of the plurality ofsecondary studies selected from the list based on the Display Protocolselected in step (d).

In an embodiment, the AnatomicalCharacteristic is selected from thegroup consisting of SPINE, CHEST, ABDOMEN, BREAST, SHOULDER, TRAPEZIUS,ARM, ELBOW, WRIST, FINGER, PELVIS, HIP, FIBULAR, KNEE, TIBULAR, ANKLE,FOOT, NECK, HEAD, TEMPOROMANDIBULAR JUNCTION, FACE, BRAIN, DENTITION,SINUS, ADRENALS, RETINA, PITUITARY, and PROSTATE.

In an embodiment, the method further includes the Study Selection Rulein step (c)(i)

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        Other.Dicom.Modality=“ParameterB”).

In an embodiment, the method further includes where in step (c)(ii) theCNN generates Primary.ImageContentBasedParameter=“ParameterC” andOther.ImageContentBasedParameter=“ParameterD”, and the Study SelectionRule in step (c)(i) and in step (c)(iii)

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(ParameterC,ParameterD) NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),        -   THEN SELECT other studies for loading

-   WHERE (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND    Other.Dicom.Modality=“ParameterB” AND

-   INTERSECTION(Primary.ImageContentBasedParameter,Other.ImageContentBasedParameter)    NOT EMPTY).

In an embodiment, the one or more ImageContentBased parameters arevertebrae. In an embodiment, the CNN identifies vertebrae in the primarystudy selected from the group consisting of L1, L2, L3, L4, L5, C1, C2,C3, C4, C5, C6, C7, Th1, Th2, Th3, Th4, Th5, Th6, Th7, Th8, Th9, Th10,Th11, and Th12. In an embodiment, the CNN is pretrained with theplurality of studies. In an embodiment, the CNN is pretrained with afirst plurality of studies where the first plurality of studies isselected based on the anatomical characteristic in the primary study. Inan embodiment, the CNN is pretrained with a first plurality of studieswhere the first plurality of studies is selected based on one or moreImageContentBased parameters identified in the primary study.

In another aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting as a ParameterA an anatomical characteristic in the primarystudy; (c) executing on a server digital data processor a render serverprogram which applies one or more Study Selection Rules to: (i) generatea list of a plurality of secondary studies based on ParameterA; (ii)generate from the list of the plurality of secondary studies one or moreImageContentBased parameters using Convolutional Neural Networks (CNN);(iii) select from the list of the plurality of secondary studies a finallist based on the one or more ImageContentBased parameters; (d)executing on the server digital data processor the render server programwhich applies one or more Protocol Selection Rules to select a DisplayProtocol, where the one or more Protocol Selection Rules are based ontwo or more parameters selected from the group consisting of one or moreDICOM parameters from the primary Study, one or more Abstract Tags fromthe primary Study, one or more DICOM parameters from the plurality ofsecondary studies, one or more Abstract Tags from the plurality ofsecondary studies and one or more ImageContentBased parameters; and (e)displaying the primary study and one or more of the plurality ofsecondary studies selected from the list based on the Display Protocolselected in step (d).

In an embodiment, the AnatomicalCharacteristic is selected from thegroup consisting of SPINE, CHEST, ABDOMEN, BREAST, SHOULDER, TRAPEZIUS,ARM, ELBOW, WRIST, FINGER, PELVIS, HIP, FIBULAR, KNEE, TIBULAR, ANKLE,FOOT, NECK, HEAD, TEMPOROMANDIBULAR JUNCTION, FACE, BRAIN, DENTITION,SINUS, ADRENALS, RETINA, PITUITARY, and PROSTATE.

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA”).

In an embodiment, the method further includes where in step (c)(ii) theCNN generates Primary.ImageContentBasedParameter=“ParameterC” andOther.ImageContentBasedParameter=“ParameterD”, and a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) and in step(c)(iii) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        INTERSECTION(ParameterC,ParameterD) NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        INTERSECTION(Primary.ImageContentBasedParameter,Other.ImageContentBasedParameter)        NOT EMPTY). p In an embodiment, the method further includes        where a ParameterB is a Modality in the primary study, where in        step (c) the one or more Study Selection Rules restrict to        studies where the Modality in the final list is equal to        ParameterB.

In an embodiment, a ParameterB is selected from the group consisting ofComputed Radiography (CR), Computer Tomography (CT), Digital Radiography(DX), Mammography (MG), Magnetic Resonance (MR), Opthalmic Photography(OP), Positron Emission Tomography (PT), Radio Fluoroscopy (RF), andX-Ray Angiography (XA).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        Other.Dicom.Modality=“ParameterB”).

In an embodiment, the method further includes where in step (c)(ii) theCNN generates Primary.ImageContentBasedParameter=“ParameterC” andOther.ImageContentBasedParameter=“ParameterD”, and a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) and in step(c)(iii) require

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(ParameterC,ParameterD) NOT EMPTY).

In an embodiment, the method further includes wherein a Study SelectionRule of the one or more Study Selection Rules requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(Primary.ImageContentBasedParameter,Other.ImageContentBasedParameter)        NOT EMPTY).

In another aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting as a ParameterA a disease based characteristic in the primarystudy; (c) executing on a server digital data processor a render serverprogram which applies one or more Study Selection Rules to: (i) generatea list of a plurality of secondary studies based on ParameterA; (ii)generate from the list of the plurality of secondary studies one or moreImageContentBased parameters using Convolutional Neural Networks (CNN);(iii) select from the list of the plurality of secondary studies a finallist based on the one or more ImageContentBased parameters;

(d) executing on the server digital data processor the render serverprogram which applies one or more Protocol Selection Rules to select aDisplay Protocol, where the one or more Protocol Selection Rules arebased on two or more parameters selected from the group consisting ofone or more DICOM parameters from the primary Study, one or moreAbstract Tags from the primary Study, one or more DICOM parameters fromthe plurality of secondary studies, one or more Abstract Tags from theplurality of secondary studies and one or more ImageContentBasedparameters; and (e) displaying the primary study and one or more of theplurality of secondary studies selected from the list based on theDisplay Protocol selected in step (d).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) requires

-   -   IF (Primary.Dicom.DiseaseCharacteristic=“ParameterA”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.DiseaseCharacteristic=“ParameterA”).

In an embodiment, the method further includes where in step (c)(ii) theCNN generates Primary.ImageContentBasedParameter=“ParameterC” andOther.ImageContentBasedParameter=“ParameterD”, and a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) and in step(c)(iii) require

-   -   IF (Primary.Dicom.DiseaseCharacteristic=“ParameterA”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.DiseaseCharacteristic=“ParameterA” AND        INTERSECTION(ParameterC,ParameterD) NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        INTERSECTION(Primary.ImageContentBasedParameter,Other.ImageContentBasedParameter)        NOT EMPTY).

In an embodiment, the method further includes where a ParameterB is aModality in the primary study, where in step (c) the one or more StudySelection Rules restrict to studies where the Modality in the final listis equal to ParameterB.

In an embodiment, a ParameterB is selected from the group consisting ofComputed Radiography (CR), Computer Tomography (CT), Digital Radiography(DX), Mammography (MG), Magnetic Resonance (MR), Opthalmic Photography(OP), Positron Emission Tomography (PT), Radio Fluoroscopy (RF), andX-Ray Angiography (XA).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        Other.Dicom.Modality=“ParameterB”).

In an embodiment, the method further includes where in step (c)(ii) theCNN generates Primary.ImageContentBasedParameter=“ParameterC” andOther.ImageContentBasedParameter=“ParameterD”, and a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) and in step(c)(iii) require

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(ParameterC,ParameterD) NOT EMPTY).

In an embodiment, the method further includes the Study Selection Rule

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(Primary.ImageContentBasedParameter,Other.ImageContentBasedParameter)        NOT EMPTY).

In another aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting as a ParameterB a Modality in the primary study and selectingas a ParameterA from the group consisting of an anatomicalcharacteristic and a disease based characteristic in the primary study;(c) executing on a server digital data processor a render server programwhich applies one or more Study Selection Rules to: (i) generate a listof a plurality of secondary studies based on ParameterA and ParamaterB;(ii) generate from the list of the plurality of secondary studies one ormore ImageContentBased parameters using Convolutional Neural Networks(CNN); (iii) select from the list of the plurality of secondary studiesa final list based on the one or more ImageContentBased parameters; (d)executing on the server digital data processor the render server programwhich applies one or more Protocol Selection Rules to select a DisplayProtocol, where the one or more Protocol Selection Rules are based ontwo or more parameters selected from the group consisting of one or moreDICOM parameters from the primary Study, one or more Abstract Tags fromthe primary Study, one or more DICOM parameters from the plurality ofsecondary studies, one or more Abstract Tags from the plurality ofsecondary studies and one or more ImageContentBased parameters; and (e)displaying the primary study and one or more of the plurality ofsecondary studies selected from the list based on the Display Protocolselected in step (d).

In an embodiment, the one or more ImageContentBased parameters arevertebrae. In an embodiment, the CNN identifies vertebrae in the primarystudy selected from the group consisting of L1, L2, L3, L4, L5, C1, C2,C3, C4, C5, C6, C7, Th1, Th2, Th3, Th4, Th5, Th6, Th7, Th8, Th9, Th10,Th11, and Th12. In an embodiment, the CNN is pretrained with theplurality of studies. In an embodiment, the CNN is pretrained with afirst plurality of studies where the first plurality of studies isselected based on the anatomical characteristic in the primary study. Inan embodiment, the CNN is pretrained with a first plurality of studieswhere the first plurality of studies is selected based on one or moreImageContentBased parameters identified in the primary study. In anembodiment, the one or more ImageContentBased parameters are vertebrae.In an embodiment, the CNN identifies vertebrae in the primary studyselected from the group consisting of L1, L2, L3, L4, L5, C1, C2, C3,C4, C5, C6, C7, Th1, Th2, Th3, Th4, Th5, Th6, Th7, Th8, Th9, Th10, Th11,and Th12. In an embodiment, the CNN is pretrained with the plurality ofstudies. In an embodiment, the CNN is pretrained with a first pluralityof studies where the first plurality of studies is selected based on theanatomical characteristic in the primary study. In an embodiment, theCNN is pretrained with a first plurality of studies where the firstplurality of studies is selected based on one or more ImageContentBasedparameters identified in the primary study. In an embodiment, the one ormore ImageContentBased parameters are vertebrae. In an embodiment, theCNN identifies vertebrae in the primary study selected from the groupconsisting of L1, L2, L3, L4, L5, C1, C2, C3, C4, C5, C6, C7, Th1, Th2,Th3, Th4, Th5, Th6, Th7, Th8, Th9, Th10, Th11, and Th12. In anembodiment, the CNN is pretrained with the plurality of studies. In anembodiment, the CNN is pretrained with a first plurality of studieswhere the first plurality of studies is selected based on the anatomicalcharacteristic in the primary study. In an embodiment, the CNN ispretrained with a first plurality of studies where the first pluralityof studies is selected based on one or more ImageContentBased parametersidentified in the primary study.

In another aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting as a ParameterA an AnatomicalCharacteristic in the primarystudy; (c) executing on a server digital data processor a render serverprogram which applies one or more Study Selection Rules to generate alist of a plurality of secondary studies based on the ParameterA and oneor more parameters selected from the group consisting of one or moreDICOM parameters from the primary Study, one or more Abstract Tags fromthe primary Study, one or more DICOM parameters from the plurality ofsecondary studies and one or more Abstract Tags from the plurality ofsecondary studies, where the one or more Study Selection Rules restrictthe plurality of secondary studies to studies of the patient selectedfrom the plurality of studies, where the one or more Study SelectionRules restrict the plurality of secondary studies to studies of theAnatomicalCharacteristic in the primary study; (d) executing on theserver digital data processor the render server program which appliesone or more Protocol Selection Rules to select a Display Protocol, wherethe one or more Protocol Selection Rules are based on two or moreparameters selected from the group consisting of one or more DICOMparameters from the primary Study, one or more Abstract Tags from theprimary Study, one or more DICOM parameters from the plurality ofsecondary studies and one or more Abstract Tags from the plurality ofsecondary studies; and (e) displaying two or more of the plurality ofsecondary studies selected from the list based on the Display Protocolselected in step (d).

In an embodiment, the AnatomicalCharacteristic is selected from thegroup consisting of SPINE, CHEST, ABDOMEN, BREAST, SHOULDER, TRAPEZIUS,ARM, ELBOW, WRIST, FINGER, PELVIS, HIP, FIBULAR, KNEE, TIBULAR, ANKLE,FOOT, NECK, HEAD, TEMPOROMANDIBULAR JUNCTION, FACE, BRAIN, DENTITION,SINUS, ADRENALS, RETINA, PITUITARY, and PROSTATE. In an embodiment, theone or more parameters include a ParameterB a Modality in the primarystudy and a ParameterY a Modality in a secondary study, where in step(c) the one or more Study Selection Rules restrict to studies whereParameterB is equal to ParameterY.

In an embodiment, the ParameterB is selected from the group consistingof Computed Radiography (CR), Computer Tomography (CT), DigitalRadiography (DX), Mammography (MG), Magnetic Resonance (MR), OpthalmicPhotography (OP), Positron Emission Tomography (PT), Radio Fluoroscopy(RF), and X-Ray Angiography (XA). In an embodiment, when the ParameterBis Computed Radiography (CR) then the Modality in the two or more of theplurality of secondary studies is selected from the group consisting ofComputer Tomography (CT), Digital Radiography (DX), Mammography (MG),Magnetic Resonance (MR), Opthalmic Photography (OP), Positron EmissionTomography (PT), Radio Fluoroscopy (RF), and X-Ray Angiography (XA).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“CR”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        (Other.Dicom.Modality=“CR”) or Other.Dicom.Modality=“CT”).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        (Other.Dicom.Modality=“ParameterZ”) AND INTERSECTION        (ParameterB, ParameterZ) NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        (Other.Dicom.Modality=“ParameterY”) AND INTERSECTION        (ParameterB, ParameterY) EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”)    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        (Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)        NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”)    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        (Other.Dicom.Modality=NOT “ParameterB” AND        INTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)        NOT EMPTY).

In an embodiment, the AnatomicalFeature is a medical condition. In anembodiment, the medical condition is selected from the group consistingof a fracture and a bleeding. In an embodiment, the AnatomicalFeature isa notation stored as meta data. In an embodiment, the AnatomicalFeatureincludes whether an organ is present in a study. In an embodiment, theParameterA is selected from meta data stored in the primary Study.

In another aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting as a ParameterA a disease based characteristic in the primarystudy; (c) executing on a server digital data processor a render serverprogram which applies one or more Study Selection Rules to generate alist of a plurality of secondary studies based on the ParameterA and oneor more parameters selected from the group consisting of one or moreDICOM parameters from the primary Study, one or more Abstract Tags fromthe primary Study, one or more DICOM parameters from the plurality ofsecondary studies and one or more Abstract Tags from the plurality ofsecondary studies, where the one or more Study Selection Rules restrictthe plurality of secondary studies to studies of the patient selectedfrom the plurality of studies, where the one or more Study SelectionRules restrict the plurality of secondary studies to studies of thedisease based characteristic in the primary study; (d) executing on theserver digital data processor the render server program which appliesone or more Protocol Selection Rules to select a Display Protocol, wherethe one or more Protocol Selection Rules are based on two or moreparameters selected from the group consisting of one or more DICOMparameters from the primary Study, one or more Abstract Tags from theprimary Study, one or more DICOM parameters from the plurality ofsecondary studies and one or more Abstract Tags from the plurality ofsecondary studies; and (e) displaying two or more of the plurality ofsecondary studies selected from the list based on the Display Protocolselected in step (d).

In an embodiment, the one or more parameters include a ParameterB aModality in the primary study and a ParameterY a Modality in a secondarystudy, where in step (c) the one or more Study Selection Rules restrictto studies where ParameterB is equal to ParameterY.

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   SELECT other secondary studies for loading WHERE        (Primary.Dicom.Modality=“ParameterB” and        (Other.Dicom.Modality=“ParameterY” AND        INTERSECTION(ParameterB,ParameterY) NOT EMPTY).

In an embodiment, the ParameterB is selected from the group consistingof Computed Radiography (CR), Computer Tomography (CT), DigitalRadiography (DX), Mammography (MG), Magnetic Resonance (MR), OpthalmicPhotography (OP), Positron Emission Tomography (PT), Radio Fluoroscopy(RF), and X-Ray Angiography (XA).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.DiseaseCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“CR”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.DiseaseCharacteristic=“ParameterA” and        (Other.Dicom.Modality=“CR” or Other.Dicom.Modality=“CT”)).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.DiseaseCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.DiseaseCharacteristic=“ParameterA” and        (Other.Dicom.Modality=“ParameterB” or        Other.Dicom.Modality=“ParameterZ”)).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.DiseaseCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.DiseaseCharacteristic=“ParameterA” and        (Other.Dicom.Modality=“ParameterY”) WHERE Intersection        (ParameterB, ParameterY) NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

IF (Primary.Dicom.DiseaseCharacteristic=“DISEASE 1” andPrimary.Dicom.Modality=“IMAGE TYPE 1”),

-   -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.DiseaseCharacteristic=“DISEASE 1” and        Other.Dicom.Modality=“IMAGE TYPE 2”).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.DiseaseCharacteristic=“SpinalDegradation” and        Primary.Dicom.Modality=“ParameterB”)    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“SpinalDegradation” and        Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)        NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.DiseaseCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”)    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.DiseaseCharacteristic=“ParameterA” and        Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)        NOT EMPTY).

In an embodiment, the AnatomicalFeature is a medical condition. In anembodiment, the medical condition is selected from the group consistingof a fracture and a bleeding. In an embodiment, the AnatomicalFeature isa notation stored as meta data. In an embodiment, the AnatomicalFeatureincludes whether an organ is present in a study. In an embodiment, theParameterA is selected from meta data stored in the primary Study.

In another aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting a ParameterA from the primary study, where the ParameterA isselected from the group consisting of an anatomical characteristic, ananatomical region imaged, and a disease based characteristic; (c)executing on a server digital data processor a render server programwhich applies one or more Study Selection Rules to generate a list of aplurality of secondary studies based on the ParameterA and one or moreparameters selected from the group consisting of one or more DICOMparameters from the primary Study, one or more Abstract Tags from theprimary Study, one or more DICOM parameters from the plurality ofsecondary studies and one or more Abstract Tags from the plurality ofsecondary studies, where the one or more Study Selection Rules restrictthe plurality of secondary studies to studies of the patient selectedfrom the plurality of studies, where the one or more Study SelectionRules restrict the plurality of secondary studies to studies of theParameterA selected;

(d) executing on the server digital data processor the render serverprogram which applies one or more Protocol Selection Rules to select aDisplay Protocol, where the one or more Protocol Selection Rules arebased on two or more parameters selected from the group consisting ofone or more DICOM parameters from the primary Study, one or moreAbstract Tags from the primary Study, one or more DICOM parameters fromthe plurality of secondary studies and one or more Abstract Tags fromthe plurality of secondary studies; and (e) displaying two or more ofthe plurality of secondary studies selected from the list based on theDisplay Protocol selected in step (d).

In an embodiment, ParameterA is the anatomical region imaged selectedfrom the group consisting of SPINE, CHEST, ABDOMEN, BREAST, SHOULDER,TRAPEZIUS, ARM, ELBOW, WRIST, FINGER, PELVIS, HIP, FIBULAR, KNEE,TIBULAR, ANKLE, FOOT, NECK, HEAD, TEMPOROMANDIBULAR JUNCTION, FACE,BRAIN, DENTITION, SINUS, ADRENALS, RETINA, PITUITARY, and PROSTATE. Inan embodiment, the one or more parameters include a ParameterB aModality in the primary study and a ParameterY a Modality in a secondarystudy, where in step (c) the one or more Study Selection Rules restrictto studies where ParameterB is equal to ParameterY.

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   SELECT other secondary studies for loading WHERE        (Primary.Dicom.Modality=“ParameterB” and        (Other.Dicom.Modality=“ParameterY” AND        INTERSECTION(ParameterB,ParameterY) NOT EMPTY).

In an embodiment, the ParameterB is selected from the group consistingof Computed Radiography (CR), Computer Tomography (CT), DigitalRadiography (DX), Mammography (MG), Magnetic Resonance (MR), OpthalmicPhotography (OP), Positron Emission Tomography (PT), Radio Fluoroscopy(RF), and X-Ray Angiography (XA).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“CR”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        Other.Dicom.Modality=“CR” or (Other.Dicom.Modality=“CT”)).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        (Other.Dicom.Modality=“ParameterB” or        Other.Dicom.Modality=“ParameterZ”) WHERE Intersection        (ParameterB, ParameterZ) EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        (Other.Dicom.Modality=“ParameterY”) AND INTERSECTION        (ParameterB, ParameterY) NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“Spine” and        Primary.Dicom.Modality=“ParameterB”)    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“Spine” and        Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)        NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”)    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)        NOT EMPTY).

In an embodiment, the AnatomicalFeature is a medical condition. In anembodiment, the medical condition is selected from the group consistingof a fracture and a bleeding. In an embodiment, the AnatomicalFeature isa notation stored as meta data. In an embodiment, the AnatomicalFeatureincludes whether an organ is present in a study. In an embodiment, theParameterA is selected from meta data stored in the primary Study.

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.DiseaseCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“CR”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.DiseaseCharacteristic=“ParameterA” and        Other.Dicom.Modality=“CR” or Other.Dicom.Modality=“CT”).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.DiseaseCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.DiseaseCharacteristic=“ParameterA” and        (Other.Dicom.Modality=“ParameterB” or        Other.Dicom.Modality=“ParameterZ”) WHERE Intersection        (ParameterB, ParameterZ) EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.DiseaseCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.DiseaseCharacteristic=“ParameterA” and        Other.Dicom.Modality=“ParameterY” WHERE Intersection        (ParameterB, ParameterY) NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.DiseaseCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.DiseaseCharacteristic=“ParameterA” and        Other.Dicom.Modality=“ParameterB”).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c) requires

-   -   IF (Primary.Dicom.DiseaseCharacteristic=“SpinalDegradation” and        Primary.Dicom.Modality=“ParameterB”)    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“SpinalDegradation” and        Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)        NOT EMPTY).

In an embodiment, the AnatomicalFeature is a medical condition.

In another aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting a ParameterA from the primary study, where the ParameterA isselected from the group consisting of an anatomical characteristic; (c)selecting ParameterB selected from the group consisting of one or moreof regions of the anatomical characteristic, parts of a skeletal systemof the anatomical characteristic, and organs of the anatomicalcharacteristic; (d) executing on a server digital data processor arender server program which applies one or more Study Selection Rules togenerate a list of a plurality of secondary studies based on theParameterA and the ParameterB, where the one or more Study SelectionRules restrict the plurality of secondary studies to studies of thepatient selected from the plurality of studies; (e) executing on theserver digital data processor the render server program which appliesone or more Protocol Selection Rules to select a Display Protocol, wherethe one or more Protocol Selection Rules are based on two or moreparameters selected from the group consisting of one or more DICOMparameters from the primary Study, one or more Abstract Tags from theprimary Study, one or more DICOM parameters from the plurality ofsecondary studies and one or more Abstract Tags from the plurality ofsecondary studies; and (f) displaying two or more of the plurality ofsecondary studies selected from the list based on the Display Protocolselected in step (e).

In an embodiment, the anatomical characteristic is selected from thegroup consisting of HEAD, EYE, EAR, NOSE, NOSTRIL, MOUTH, LIP, PHILTRUM,JAW, MANDIBLE, GINGIVA, TOOTH, TONGUE, THROAT, LARYNGEAL PROMINENCE,VERTEBRAL COLUMN, SCAPULA, HUMERUS, ELBOW, RADIUS, ULNA, CARPUS,METACARPUS, PHALANGES, THUMB, NAILS, THORAX, BREAST, ABDOMEN, PENIS,SCROTUM, VULVA, LEG, FEMUR, KNEE, PATELLA, TIBIA, SURA, TALOCRURALREGION, METATARSUS, PHALANGES PROXIMALES, PHALANGES MEDIAE, ANDPHALANGES DISTALES.

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (d) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.RegionAnatomicalCharacteristic=“ParameterB”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        Other.Dicom.OrganAnatomicalCharacteristic=“ParameterC”).

In an embodiment, one or both the ParameterA and the ParameterB areselected from meta data stored in the primary Study.

In another aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies, wherethe primary Study is an image measured after the patient has a contrastagent administered; (b) selecting a ParameterA anAnatomicalCharacteristic from the primary study; (c) selectingParameterB a Modality from the primary study; (d) executing on a serverdigital data processor a render server program which applies one or moreStudy Selection Rules to generate a list of a plurality of secondarystudies based on the ParameterA and the ParameterB, where the one ormore Study Selection Rules restrict the plurality of secondary studiesto studies of the patient before the contrast agent was administeredselected from the plurality of studies; (e) executing on the serverdigital data processor the render server program which applies one ormore Protocol Selection Rules to select a Display Protocol, where theone or more Protocol Selection Rules are based on two or more parametersselected from the group consisting of one or more DICOM parameters fromthe primary Study, one or more Abstract Tags from the primary Study, oneor more DICOM parameters from the plurality of secondary studies and oneor more Abstract Tags from the plurality of secondary studies; and (f)displaying two or more of the plurality of secondary studies selectedfrom the list based on the Display Protocol selected in step (e).

In an embodiment, the AnatomicalCharacteristic is selected from thegroup consisting of SPINE, CHEST, ABDOMEN, BREAST, SHOULDER, TRAPEZIUS,ARM, ELBOW, WRIST, FINGER, PELVIS, HIP, FIBULAR, KNEE, TIBULAR, ANKLE,FOOT, NECK, HEAD, TEMPOROMANDIBULAR JUNCTION, FACE, BRAIN, DENTITION,SINUS, ADRENALS, RETINA, PITUITARY, and PROSTATE. In an embodiment, theParameterB is selected from the group consisting of Computed Radiography(CR), Computer Tomography (CT), Digital Radiography (DX), Mammography(MG), Magnetic Resonance (MR), Opthalmic Photography (OP), PositronEmission Tomography (PT), Radio Fluoroscopy (RF), and X-Ray Angiography(XA).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (d) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“CR”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        Other.Dicom.Modality=“CR” or Other.Dicom.Modality=“CT”).

In an embodiment, one or both the ParameterA and the ParameterB areselected from meta data stored in the primary Study.

In another aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting a ParameterA an AnatomicalCharacteristic from the primarystudy; (c) selecting ParameterB selected from the group consisting ofone or more anatomical characteristics of the AnatomicalCharacteristic,parts of a skeletal system of the AnatomicalCharacteristic, organs ofthe AnatomicalCharacteristic and Modality of the primary Study; (d)executing on a server digital data processor a render server programwhich applies one or more Study Selection Rules to generate a list of aplurality of secondary studies based on the ParameterA and theParameterB, where the one or more Study Selection Rules restrict theplurality of secondary studies to studies of the patient selected fromthe plurality of studies; (e) executing on the server digital dataprocessor the render server program which applies one or more ProtocolSelection Rules to select a Display Protocol, where the one or moreProtocol Selection Rules are based on two or more parameters selectedfrom the group consisting of one or more DICOM parameters from theprimary Study, one or more Abstract Tags from the primary Study, one ormore DICOM parameters from the plurality of secondary studies and one ormore Abstract Tags from the plurality of secondary studies; and (f)displaying two or more of the plurality of secondary studies selectedfrom the list based on the Display Protocol selected in step (e).

In an embodiment, the AnatomicalCharacteristic is selected from thegroup consisting of SPINE, CHEST, ABDOMEN, BREAST, SHOULDER, TRAPEZIUS,ARM, ELBOW, WRIST, FINGER, PELVIS, HIP, FIBULAR, KNEE, TIBULAR, ANKLE,FOOT, NECK, HEAD, TEMPOROMANDIBULAR JUNCTION, FACE, BRAIN, DENTITION,SINUS, ADRENALS, RETINA, PITUITARY, and PROSTATE. In an embodiment, theParameterB is selected from the group consisting of Computed Radiography(CR), Computer Tomography (CT), Digital Radiography (DX), Mammography(MG), Magnetic Resonance (MR), Opthalmic Photography (OP), PositronEmission Tomography (PT), Radio Fluoroscopy (RF), and X-Ray Angiography(XA).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (d) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“SPINE” and        Primary.Dicom.AnatomicalFeature=“ParameterB”)    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“SPINE” and        Other.Dicom.AnatomicalFeature=“ParameterB” AND        INTERSECTION(Primary.AnatomicalFeature,Other.AnatomicalFeature)        NOT EMPTY).

In an embodiment, the ParameterA is selected from meta data stored inthe primary Study.

In another aspect, there is provided a including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting a ParameterA from the primary study, where the ParameterA isselected from the group consisting of an anatomical characteristic, anda disease based characteristic; (c) selecting ParameterB from the groupconsisting of one or more anatomical features of the ParameterA, partsof a skeletal system of the ParameterA, organs of the ParameterA, and aModality of the primary Study; (d) executing on a server digital dataprocessor a render server program which applies one or more StudySelection Rules to generate a list of a plurality of secondary studiesbased on the ParameterA and the ParameterB, where the one or more StudySelection Rules restrict the plurality of secondary studies to studiesof the patient selected from the plurality of studies; (e) executing onthe server digital data processor the render server program whichapplies one or more Protocol Selection Rules to select a DisplayProtocol, where the one or more Protocol Selection Rules are based ontwo or more parameters selected from the group consisting of one or moreDICOM parameters from the primary Study, one or more Abstract Tags fromthe primary Study, one or more DICOM parameters from the plurality ofsecondary studies and one or more Abstract Tags from the plurality ofsecondary studies; and (f) displaying two or more of the plurality ofsecondary studies selected from the list based on the Display Protocolselected in step (e).

In an embodiment, the anatomical characteristic is selected from thegroup consisting of SPINE, CHEST, ABDOMEN, BREAST, SHOULDER, TRAPEZIUS,ARM, ELBOW, WRIST, FINGER, PELVIS, HIP, FIBULAR, KNEE, TIBULAR, ANKLE,FOOT, NECK, HEAD, TEMPOROMANDIBULAR JUNCTION, FACE, BRAIN, DENTITION,SINUS, ADRENALS, RETINA, PITUITARY, and PROSTATE. In an embodiment, theParameterB is a Modality in the primary study and a ParameterY aModality in a secondary study, where in step (d) the one or more StudySeletion Rules restrict to studies where ParameterB is equal toParameterY.

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (d) requires

-   -   SELECT other secondary studies for loading WHERE        (Primary.Dicom.Modality=“ParameterB” and        (Other.Dicom.Modality=“ParameterY” AND        INTERSECTION(ParameterB,ParameterY) NOT EMPTY).

In an embodiment, the ParameterB is a Modality selected from the groupconsisting of Computed Radiography (CR), Computer Tomography (CT),Digital Radiography (DX), Mammography (MG), Magnetic Resonance (MR),Opthalmic Photography (OP), Positron Emission Tomography (PT), RadioFluoroscopy (RF), and X-Ray Angiography (XA).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (d) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        Other.Dicom.Modality=“ParameterB”).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (d) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        Other.Dicom.Modality=“ParameterY” AND INTERSECTION (ParameterB,        ParameterY) NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (d) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”)    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        (Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)        NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (d) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA”)    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        INTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)        NOT EMPTY).

In another aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting as a ParameterA an anatomical characteristic imaged; (c)selecting ParameterB from the group consisting of one or more anatomicalfeatures of the ParameterA, parts of a skeletal system of theParameterA, organs of the ParameterA, and a Modality of the primaryStudy; (d) executing on a server digital data processor a render serverprogram which applies one or more Study Selection Rules to generate alist of a plurality of secondary studies based on the ParameterA and theParameterB, where the one or more Study Selection Rules restrict theplurality of secondary studies to studies of the patient selected fromthe plurality of studies; (e) executing on the server digital dataprocessor the render server program which applies one or more ProtocolSelection Rules to select a Display Protocol, where the one or moreProtocol Selection Rules are based on two or more parameters selectedfrom the group consisting of one or more DICOM parameters from theprimary Study, one or more Abstract Tags from the primary Study, one ormore DICOM parameters from the plurality of secondary studies and one ormore Abstract Tags from the plurality of secondary studies; and (f)displaying two or more of the plurality of secondary studies selectedfrom the list based on the Display Protocol selected in step (e).

In an embodiment, the anatomical characteristic is selected from thegroup consisting of SPINE, CHEST, ABDOMEN, BREAST, SHOULDER, TRAPEZIUS,ARM, ELBOW, WRIST, FINGER, PELVIS, HIP, FIBULAR, KNEE, TIBULAR, ANKLE,FOOT, NECK, HEAD, TEMPOROMANDIBULAR JUNCTION, FACE, BRAIN, DENTITION,SINUS, ADRENALS, RETINA, PITUITARY, and PROSTATE. In an embodiment, theParameterB is a Modality in the primary study and a ParameterY aModality in a secondary study, where in step (d) the one or more StudySelection Rules restrict to studies where ParameterB is equal toParameterY.

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (d) requires

-   -   SELECT other secondary studies for loading WHERE        (Primary.Dicom.Modality=“ParameterB” and        (Other.Dicom.Modality=“ParameterY” AND        INTERSECTION(ParameterB,ParameterY) NOT EMPTY).

In an embodiment, the ParameterB is a Modality selected from the groupconsisting of Computed Radiography (CR), Computer Tomography (CT),Digital Radiography (DX), Mammography (MG), Magnetic Resonance (MR),Opthalmic Photography (OP), Positron Emission Tomography (PT), RadioFluoroscopy (RF), and X-Ray Angiography (XA).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (d) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        Other.Dicom.Modality=“ParameterB”).

In an embodiment, the method further includes the Study Selection Rule

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        Other.Dicom.Modality=“ParameterY” AND INTERSECTION (ParameterB,        ParameterY) NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (d) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”)    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        (Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)        NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (d) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA”)    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        INTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)        NOT EMPTY).

In another aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting as a ParameterA a disease based characteristic; (c) selectingParameterB from the group consisting of one or more anatomical featuresof the ParameterA, parts of a skeletal system of the ParameterA, organsof the ParameterA, and a Modality of the primary Study; (d) executing ona server digital data processor a render server program which appliesone or more Study Selection Rules to generate a list of a plurality ofsecondary studies based on the ParameterA and the ParameterB, where theone or more Study Selection Rules restrict the plurality of secondarystudies to studies of the patient selected from the plurality ofstudies; (e) executing on the server digital data processor the renderserver program which applies one or more Protocol Selection Rules toselect a Display Protocol, where the one or more Protocol SelectionRules are based on two or more parameters selected from the groupconsisting of one or more DICOM parameters from the primary Study, oneor more Abstract Tags from the primary Study, one or more DICOMparameters from the plurality of secondary studies and one or moreAbstract Tags from the plurality of secondary studies; and (f)displaying two or more of the plurality of secondary studies selectedfrom the list based on the Display Protocol selected in step (e).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (d) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA”)    -   THEN SELECT other secondary studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        INTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)        NOT EMPTY).

In another aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting as a ParameterA an anatomical characteristic in the primarystudy; (c) executing on a server digital data processor a render serverprogram which applies one or more Study Selection Rules to: (i) generatea list of a plurality of secondary studies based on ParameterA; (ii)generate from the list of the plurality of secondary studies one or moreImageContentBased parameters using Convolutional Neural Networks (CNN);(iii) select from the list of the plurality of secondary studies a finallist based on the one or more ImageContentBased parameters; (d)executing on the server digital data processor the render server programwhich applies one or more Protocol Selection Rules to select a DisplayProtocol, where the one or more Protocol Selection Rules are based ontwo or more parameters selected from the group consisting of one or moreDICOM parameters from the primary Study, one or more Abstract Tags fromthe primary Study, one or more DICOM parameters from the plurality ofsecondary studies, one or more Abstract Tags from the plurality ofsecondary studies and one or more ImageContentBased parameters; and (e)displaying the primary study and one or more of the plurality ofsecondary studies selected from the list based on the Display Protocolselected in step (d).

In an embodiment, the AnatomicalCharacteristic is selected from thegroup consisting of SPINE, CHEST, ABDOMEN, BREAST, SHOULDER, TRAPEZIUS,ARM, ELBOW, WRIST, FINGER, PELVIS, HIP, FIBULAR, KNEE, TIBULAR, ANKLE,FOOT, NECK, HEAD, TEMPOROMANDIBULAR JUNCTION, FACE, BRAIN, DENTITION,SINUS, ADRENALS, RETINA, PITUITARY, and PROSTATE.

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA”).

In an embodiment, the method further includes where in step (c)(ii) theCNN generates Primary.ImageContentBasedParameter=“ParameterC” andOther.ImageContentBasedParameter=“ParameterD”, and a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) and in step(c)(iii) require

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        INTERSECTION(ParameterC,ParameterD) NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        INTERSECTION(Primary.ImageContentBasedParameter,Other.ImageContentBasedParameter)        NOT EMPTY).

In an embodiment, the method further includes where a ParameterB is aModality in the primary study, where in step (c) the one or more StudySelection Rules restrict to studies where the Modality in the final listis equal to ParameterB.

In an embodiment, a ParameterB is selected from the group consisting ofComputed Radiography (CR), Computer Tomography (CT), Digital Radiography(DX), Mammography (MG), Magnetic Resonance (MR), Opthalmic Photography(OP), Positron Emission Tomography (PT), Radio Fluoroscopy (RF), andX-Ray Angiography (XA).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        Other.Dicom.Modality=“ParameterB”).

In an embodiment, the method further includes where in step (c)(ii) theCNN generates Primary.ImageContentBasedParameter=“ParameterC” andOther.ImageContentBasedParameter=“ParameterD”, and a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) and in step(c)(iii) require

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(ParameterC,ParameterD) NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) and in step(c)(iii) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(Primary.ImageContentBasedParameter,Other.ImageContentBasedParameter)        NOT EMPTY).

In an embodiment, the one or more ImageContentBased parameters arevertebrae. In an embodiment, the CNN identifies vertebrae in the primarystudy selected from the group consisting of L1, L2, L3, L4, L5, C1, C2,C3, C4, C5, C6, C7, Th1, Th2, Th3, Th4, Th5, Th6, Th7, Th8, Th9, Th10,Th11, and Th12. In an embodiment, the CNN is pretrained with theplurality of studies. In an embodiment, the CNN is pretrained with afirst plurality of studies where the first plurality of studies isselected based on the anatomical characteristic in the primary study. Inan embodiment, the CNN is pretrained with a first plurality of studieswhere the first plurality of studies is selected based on one or moreImageContentBased parameters identified in the primary study.

In another aspect, there is providerd a method including: (a) selectinga primary Study of a patient selected from a plurality of studies; (b)selecting as a ParameterA a disease based characteristic in the primarystudy; (c) executing on a server digital data processor a render serverprogram which applies one or more Study Selection Rules to: (i) generatea list of a plurality of secondary studies based on ParameterA; (ii)generate from the list of the plurality of secondary studies one or moreImageContentBased parameters using Convolutional Neural Networks (CNN);(iii) select from the list of the plurality of secondary studies a finallist based on the one or more ImageContentBased parameters; (d)executing on the server digital data processor the render server programwhich applies one or more Protocol Selection Rules to select a DisplayProtocol, where the one or more Protocol Selection Rules are based ontwo or more parameters selected from the group consisting of one or moreDICOM parameters from the primary Study, one or more Abstract Tags fromthe primary Study, one or more DICOM parameters from the plurality ofsecondary studies, one or more Abstract Tags from the plurality ofsecondary studies and one or more ImageContentBased parameters; and (e)displaying the primary study and one or more of the plurality ofsecondary studies selected from the list based on the Display Protocolselected in step (d).

In an embodiment, the method further including where a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) requires

-   -   IF (Primary.Dicom.DiseaseCharacteristic=“ParameterA”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.DiseaseCharacteristic=“ParameterA”).

In an embodiment, the method further includes where in step (c)(ii) theCNN generates Primary.ImageContentBasedParameter=“ParameterC” andOther.ImageContentBasedParameter=“ParameterD”, and a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) and in step(c)(iii) require

-   -   IF (Primary.Dicom.DiseaseCharacteristic=“ParameterA”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.DiseaseCharacteristic=“ParameterA” AND        INTERSECTION(ParameterC,ParameterD) NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        INTERSECTION(Primary.ImageContentBasedParameter,Other.ImageContentBasedParameter)        NOT EMPTY).

In an embodiment, the method further includes where a ParameterB is aModality in the primary study, where in step (c) the one or more StudySelection Rules restrict to studies where the Modality in the final listis equal to ParameterB.

In an embodiment, a ParameterB is selected from the group consisting ofComputed Radiography (CR), Computer Tomography (CT), Digital Radiography(DX), Mammography (MG), Magnetic Resonance (MR), Opthalmic Photography(OP), Positron Emission Tomography (PT), Radio Fluoroscopy (RF), andX-Ray Angiography (XA).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) require

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” and        Other.Dicom.Modality=“ParameterB”).

In an embodiment, the method further including where in step (c)(ii) theCNN generates Primary.ImageContentBasedParameter=“ParameterC” andOther.ImageContentBasedParameter=“ParameterD”, and a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) and in step(c)(iii) require

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other studies for loading    -   WHERE (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(ParameterC,ParameterD) NOT EMPTY).

In an embodiment, the method further includes where a Study SelectionRule of the one or more Study Selection Rules in step (c)(i) requires

-   -   IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA” and        Primary.Dicom.Modality=“ParameterB”),    -   THEN SELECT other studies for loading WHERE        (Other.Dicom.AnatomicalCharacteristic=“ParameterA” AND        Other.Dicom.Modality=“ParameterB” AND        INTERSECTION(Primary.ImageContentBasedParameter,Other.ImageContentBasedParameter)        NOT EMPTY).

In another aspect, there is provided a method including: (a) selecting aprimary Study of a patient selected from a plurality of studies; (b)selecting as a ParameterB a Modality in the primary study and selectingas a ParameterA from the group consisting of an anatomicalcharacteristic and a disease based characteristic in the primary study;(c) executing on a server digital data processor a render server programwhich applies one or more Study Selection Rules to: (i) generate a listof a plurality of secondary studies based on ParameterA and ParamaterB;(ii) generate from the list of the plurality of secondary studies one ormore ImageContentBased parameters using Convolutional Neural Networks(CNN); (iii) select from the list of the plurality of secondary studiesa final list based on the one or more ImageContentBased parameters; (d)executing on the server digital data processor the render server programwhich applies one or more Protocol Selection Rules to select a DisplayProtocol, where the one or more Protocol Selection Rules are based ontwo or more parameters selected from the group consisting of one or moreDICOM parameters from the primary Study, one or more Abstract Tags fromthe primary Study, one or more DICOM parameters from the plurality ofsecondary studies, one or more Abstract Tags from the plurality ofsecondary studies and one or more ImageContentBased parameters; and (e)displaying the primary study and one or more of the plurality ofsecondary studies selected from the list based on the Display Protocolselected in step (d).

What is claimed is:
 1. A method comprising: (A) receiving at a serverfrom a client computer a primary study of a patient, the servercomprising a server digital data processor, and the client computercomprising a client digital data processor; (B) selecting a ParameterAfrom the primary study, where the ParameterA is selected from the groupconsisting of a disease based characteristic an anatomicalcharacteristic and a patient name, where the primary study is measuredusing a Modality; (C) executing on the server one or more render serverprograms which apply one or more Study Selection Rules to generate alist of a plurality of secondary studies, where the one or more StudySelection Rules restrict the plurality of secondary studies to studiesof the patient with the Modality; (D) executing on the server digitaldata processor the one or more render server programs which apply one ormore Protocol Selection Rules to select a Display Protocol, where theone or more Protocol Selection Rules are based on two or more parametersselected from the group consisting of one or more DICOM parameters fromthe primary Study, one or more Abstract Tags from the primary Study, oneor more DICOM parameters from the plurality of secondary studies and oneor more Abstract Tags from the plurality of secondary studies; and (E)sending from the server to the client computer the Display Protocolselected in step (D) and the list generated in step (C).
 2. The methodof claim 1, where the anatomical characteristic is selected from thegroup consisting of SPINE, CHEST, ABDOMEN, BREAST, SHOULDER, TRAPEZIUS,ARM, ELBOW, WRIST, FINGER, PELVIS, HIP, FIBULAR, KNEE, TIBULAR, ANKLE,FOOT, NECK, HEAD, TEMPOROMANDIBULAR JUNCTION, FACE, BRAIN, DENTITION,SINUS, ADRENALS, RETINA, PITUITARY, and PROSTATE.
 3. The method of claim1, where the Modality is selected from the group consisting of an X-Rayimage, an X-Ray Angiography, a Computer Tomography scan, a MagneticResonance Image, a Digital Radiography, a Mammography scan, OpthalmicPhotography, Radiofluoroscopy scan, a Positron Emission Tomography scan,a vascular visualization, a histological image, a confocal microscope, a3D ultrasound, and an ultrasound.
 4. The method of claim 1, furthercomprising where a Study Selection Rule of the one or more StudySelection Rules in step (C) requires SELECT other secondary studies forloading WHERE (Primary.Dicom.Modality=“ParameterB” and(Other.Dicom.Modality=“ParameterB”.
 5. The method of claim 1, furthercomprising where a Study Selection Rule of the one or more StudySelection Rules in step (C) requires IF(Primary.Dicom.AnatomicalCharacteristic=“ParameterA” andPrimary.Dicom.Modality=“ParameterB”), THEN SELECT other secondarystudies for loading WHERE(Other.Dicom.AnatomicalCharacteristic=“ParameterA” andOther.Dicom.Modality=“ParameterB”).
 6. The method of claim 1, furthercomprising where a Study Selection Rule of the one or more StudySelection Rules in step (C) requires IF(Primary.Dicom.AnatomicalCharacteristic=“ParameterA” andPrimary.Dicom.Modality=“ParameterB”), THEN SELECT other secondarystudies for loading WHEREOther.Dicom.AnatomicalCharacteristic=“ParameterC” andOther.Dicom.Modality=“ParameterB”.
 7. The method of claim 1, furthercomprising where a Study Selection Rule of the one or more StudySelection Rules in step (C) requires IF(Primary.Dicom.AnatomicalCharacteristic=“ParameterA” andPrimary.Dicom.Modality=“ParameterB”) THEN SELECT other secondary studiesfor loading WHERE (Other.Dicom.AnatomicalCharacteristic=“ParameterA” andOther.Dicom.Modality=“ParameterB” ANDINTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)NOT EMPTY).
 8. The method of claim 1, further comprising where a StudySelection Rule of the one or more Study Selection Rules in step (C)requires IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA”) THENSELECT other secondary studies for loading WHERE(Other.Dicom.AnatomicalCharacteristic=“ParameterA” ANDINTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)NOT EMPTY).
 9. The method of claim 1, further comprising where theDisplay Protocol is selected from a plurality of available DisplayProtocols.
 10. A method comprising: (A) receiving at a server from aclient computer a ParameterA, the server comprising a server digitaldata processor, and the client computer comprising a client digital dataprocessor; (B) selecting a primary study from a plurality of studiesbased on the ParameterA, where the ParameterA is selected from the groupconsisting of a disease based characteristic an anatomicalcharacteristic and a patient name, where the primary study is measuredusing a Modality; (C) executing on the server one or more render serverprograms which apply one or more Study Selection Rules to generate alist of a plurality of secondary studies, where the one or more StudySelection Rules restrict the plurality of secondary studies to studiesof the patient with the Modality from the plurality of studies; (D)executing on the server digital data processor the one or more renderserver programs which apply one or more Protocol Selection Rules toselect a Display Protocol, where the one or more Protocol SelectionRules are based on one or more parameters selected from the groupconsisting of one or more DICOM parameters from the primary Study, oneor more Abstract Tags from the primary Study, one or more DICOMparameters from the plurality of secondary studies and one or moreAbstract Tags from the plurality of secondary studies; and (E) sendingfrom the server to the client computer the Display Protocol selected instep (D) and the list generated in step (C).
 11. The method of claim 10,further comprising where the Display Protocol is selected from aplurality of available Display Protocols.
 12. The method of claim 10,where the anatomical characteristic is selected from the groupconsisting of SPINE, CHEST, ABDOMEN, BREAST, SHOULDER, TRAPEZIUS, ARM,ELBOW, WRIST, FINGER, PELVIS, HIP, FIBULAR, KNEE, TIBULAR, ANKLE, FOOT,NECK, HEAD, TEMPOROMANDIBULAR JUNCTION, FACE, BRAIN, DENTITION, SINUS,ADRENALS, RETINA, PITUITARY, and PROSTATE.
 13. The method of claim 10,where the Modality is selected from the group consisting of an X-Rayimage, an X-Ray Angiography, a Computer Tomography scan, a MagneticResonance Image, a Digital Radiography, a Mammography scan, OpthalmicPhotography, Radiofluoroscopy scan, a Positron Emission Tomography scan,a vascular visualization, a histological image, a confocal microscope, a3D ultrasound, and an ultrasound.
 14. The method of claim 10, furthercomprising where a Study Selection Rule of the one or more StudySelection Rules in step (C) requires SELECT other secondary studies forloading WHERE (Primary.Dicom.Modality=“ParameterB” and(Other.Dicom.Modality=“ParameterB”.
 15. The method of claim 10, furthercomprising where a Study Selection Rule of the one or more StudySelection Rules in step (C) requires IF(Primary.Dicom.AnatomicalCharacteristic=“ParameterA” andPrimary.Dicom.Modality=“ParameterB”), THEN SELECT other secondarystudies for loading WHERE(Other.Dicom.AnatomicalCharacteristic=“ParameterA” andOther.Dicom.Modality=“ParameterB”).
 16. The method of claim 10, furthercomprising where a Study Selection Rule of the one or more StudySelection Rules in step (C) requires IF(Primary.Dicom.AnatomicalCharacteristic=“ParameterA” andPrimary.Dicom.Modality=“ParameterB”), THEN SELECT other secondarystudies for loading WHEREOther.Dicom.AnatomicalCharacteristic=“ParameterC” andOther.Dicom.Modality=“ParameterB”.
 17. The method of claim 10, furthercomprising where a Study Selection Rule of the one or more StudySelection Rules in step (C) requires IF(Primary.Dicom.AnatomicalCharacteristic=“ParameterA” andPrimary.Dicom.Modality=“ParameterB”) THEN SELECT other secondary studiesfor loading WHERE (Other.Dicom.AnatomicalCharacteristic=“ParameterA” andOther.Dicom.Modality=“ParameterB” ANDINTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)NOT EMPTY).
 18. The method of claim 10, further comprising where a StudySelection Rule of the one or more Study Selection Rules in step (C)requires IF (Primary.Dicom.AnatomicalCharacteristic=“ParameterA”) THENSELECT other secondary studies for loading WHERE(Other.Dicom.AnatomicalCharacteristic=“ParameterA” ANDINTERSECTION(Primary.Dicom.AnatomicalFeature,Other.Dicom.AnatomicalFeature)NOT EMPTY).
 19. A method comprising: (A) receiving at a server from aclient computer a ParameterA, the server comprising a server digitaldata processor, and the client computer comprising a client digital dataprocessor; (B) selecting a primary study from a plurality of studiesbased on the ParameterA, where the ParameterA is selected from the groupconsisting of a disease based characteristic an anatomicalcharacteristic and a patient name, where the primary study is measuredusing a Modality; (C) executing on the server one or more render serverprograms which apply one or more Study Selection Rules to generate alist of a plurality of secondary studies, where the one or more StudySelection Rules restrict the plurality of secondary studies to studiesof the patient with the Modality from the plurality of studies; (D)executing on the server digital data processor the one or more renderserver programs which apply one or more Protocol Selection Rules toselect a Display Protocol, where the one or more Protocol SelectionRules are based on two or more parameters selected from the groupconsisting of one or more DICOM parameters from the primary Study, oneor more Abstract Tags from the primary Study, one or more DICOMparameters from the plurality of secondary studies and one or moreAbstract Tags from the plurality of secondary studies; and (E) sendingfrom the server to the client computer the Display Protocol selected instep (D) and the list generated in step (C).
 20. The method of claim 19,where the Modality is selected from the group consisting of an X-Rayimage, an X-Ray Angiography, a Computer Tomography scan, a MagneticResonance Image, a Digital Radiography, a Mammography scan, OpthalmicPhotography, Radiofluoroscopy scan, a Positron Emission Tomography scan,a vascular visualization, a histological image, a confocal microscope, a3D ultrasound and an ultrasound.